National Library of Australia Staff Papers, 1997

Font Size:  Small  Medium  Large

The Dublin Core:
Simplicity or complexity?

Paper given by Dr. Warwick Cathro, Assistant Director-General, Services to Libraries Division at the ALIA Seminar on Metadata, Sydney, 30 July 1997

Introduction

"After 2 years we still haven't figured out where the Dublin Core fits into the resource discovery scheme.... [We need to determine] exactly what it is we want the Dublin Core to do ... [is it] a simple metadata scheme for web authors [or is it] a machine interoperable cataloging scheme"? This question was posed earlier this year by Carl Lagoze, one of the developers of the Dublin Core metadata standard.

So what is the purpose of the Dublin Core standard? The commonly accepted view is that it is about improving retrieval for users. From our own experiences with Web search engines, we often think of metadata as improving search precision. To take an example of search failure which I encountered on Saturday: a search using ANZWERS (a pretty good search engine, in my view) on the words "privatisation" or "privatization" limited to the Australian government (gov.au) domain produced 125,771 matches.

Clearly, search engines can return a lot of irrelevant information because they have no means (or very few means) of distinguishing between important and incidental words in document texts. If we could target our searches onto words which are used as significant terms, we could achieve an enormous improvement in precision.

If we could target searches onto words or phrases that identify their correct role, we would also improve precision. For example, we could retrieve just those resources where "Green" is the name of the author, without retrieving resources about green peas or environmental issues.

Metadata can also assist in improving search recall - that is, in retrieving information resources that would otherwise be missed. For example, relevant information can be missed because sites contain types of resource in addition to HTML text, such as images, databases, and PDF documents.

The use of metadata, combined with a better harvesting process, can also improve search recall. It is known that most harvesting engines do not index every page on a site, but often only the top two or three hierarchical levels. Thus, these engines miss significant documents which, on larger and more complex sites, may be located in lower levels of the hierarchy. A better harvesting process would gather metadata from a repository created locally from a complete coverage of the local site. The data in this repository could then be gathered regularly by the harvesting engine.

The Dublin Core metadata element set was developed during 1995 and 1996 as a response to these needs to improve retrieval of information resources, especially on the World Wide Web. Dublin Core is being developed as a generic metadata standard for use by libraries, archives, government and other publishers of online information. At present, the standard consists of a set of elements to describe basic document like objects. The standards for the syntax of Dublin Core metadata (that is, the way that it is structured and expressed, as opposed to its content) are still under development. The syntax is subject to change, as new standards are being developed by the World Wide Web Consortium (W3C). In particular, the new standards embodied in HTML 4.0 should affect the metadata syntax.

The Minimalist/Structuralist issue

It was clear at the fourth Dublin Core workshop in Canberra in March that the development of this standard has reached something of a crossroads. As somebody commented at the time, this may be because the easy part (the definition of the core fields) has been completed. The key issue at the workshop was the extent to which the structure of the Dublin Core standard (until then, just a set of 15 fields) should be elaborated. This is the so-called Minimalist/Structuralist debate.

For an explanation of the Minimalist/Structuralist issue, I commend the article by Stuart Weibel in the June 1997 issue of D-Lib magazine (The 4th Dublin Core metadata workshop report). To quote from this article:

"The Minimalist point of view reflects a strong commitment to the notion that DC's primary motivating characteristic is its simplicity. This simplicity is important both for creation of metadata (for example, by authors unschooled in the cataloging arts) and for the use of metadata by tools (for example, indexing harvesters, which will probably not make use of detailed qualifiers or encoding schemes). The goal of semantic interoperability across communities can only be achieved if there is a simple core of elements that are understood to mean the same thing in every case."

Weibel went on to contrast the contending viewpoint. "The Structuralists as a group accept the danger of [variability] in exchange for the greater flexibility of a formal means of extending or qualifying elements such that they can be made more useful for the needs of a particular community.... The underlying assumption of those who would deploy qualifiers is that the added structure and richness that they can provide can be used to good effect to enhance discovery. Every decision to deploy qualifiers should be measured against the question "Will this qualifier improve discovery?"

In this context, it is necessary to explain that there are three kinds of qualifiers. One kind, known as TYPE, refines the meaning of the field. Thus, "personal" and "corporate" are TYPEs which, if present, narrow the meaning of the CREATOR field. (This is usually expressed in so-called dot notation, as "Creator.Personal" or "Creator.Corporate"). Another kind of qualifier, known as SCHEME, explains the meaning of the data contained in the field. For example, "LCSH" is a SCHEME which helps to interpret the content of the SUBJECT field. These qualifiers - SCHEME and TYPE, along with a third one which denotes the language of the content of the field - are known collectively as the "Canberra Qualifiers".

The Clarke criticisms

Recently, Roger Clarke published on the Web a paper (Beyond the Dublin Core: rich meta-data and convenience-of-use are compatible after all) which criticised the Dublin Core standard for excessive simplicity and lack of precision (this paper may be found at http://www.anu.edu.au/people/ Roger.Clarke/II/DublinCore.html Roger.Clarke/II/DublinCore.html). To quote from his paper:

"The proponents' priorities have been expressly oriented towards simplicity, and away from sophisticated structures. It is implicit in their approach that the two are incompatible. The ... desire for simplicity has resulted in a mechanism that is incapable of representing the richness of the real-world challenges that present themselves ... a richer, more sophisticated model need not be uncomfortable or inconvenient."

More specifically, this paper criticised the Dublin Core framework for shortcomings such as:

  • its lack of precision;
  • the incompleteness of its element set;
  • its lack of structure, or relationships between elements;
  • its unclear scope (what are "document-like objects"?); and
  • its failure to deal with data about the metadata.

I have some sympathies with Roger's views, although I believe that some of his concerns are misplaced. For one thing, developments over the last few months are working to improve both the precision of the definition of Dublin Core elements and some of the weaknesses in structure and completeness. Also, it is important to remember that Dublin Core is indeed meant to be a core. Because of the basic nature of the Dublin Core element set, many creators of metadata will either require extensions beyond that set to support searching on specialised data elements (e.g. for geospatial data). However, people who are searching across a range of specialised resources will still be able to benefit from improvements in search performance if they are searching on the core or common elements.

A compromise between minimalism and structuralism

At the fourth Dublin Core Workshop, a report from one of the discussion groups set out the Minimalist position that virtually every one of the 15 Dublin Core elements should contain unqualified free text. That is, there should be neither TYPEs or SCHEMEs in Dublin Core. There should be no formal constraints on the data content through adherence to particular standards or lists of authorised values. I guess this group would have made one exception: the field called RESOURCE IDENTIFIER should contain data which is formulated according to some recognised scheme, such as Uniform Resource Locator or Uniform Resource Name and, at least at present, the default scheme should be the Uniform Resource Locator (URL). But, according to this statement of the Minimalist position, everything else should be unformatted free text.

The view of those who favour qualifiers such as SCHEMEs can be illustrated by a field such as SUBJECT. This view says that you can, if you wish, put free text or keywords into the SUBJECT field. But you should also have the right, if you wish, to indicate that the keywords, phrases or codes in this field conform to a particular scheme, which might be a thesaurus (LCSH, MeSH, Art and Architecture Thesaurus, etc.) or might be a classification scheme (DDC, LC etc). To support this approach, there would need to be a metadata registry system to document the schemes. This issue is dealt with in more detail below.

The SUBJECT example suggests a general approach to a compromise, based on the following principles. Structure should not be imposed on the minimalists. Equally, minimalism should not be forced on the structuralists. The key to compromise, at least in most cases, appears to be to agree that the default situation is that the element requires no structure or qualifier. Thus, for SUBJECT, if you do not use any qualifier, then you are using free text keywords or phrases. But, if you want to flag that you are using authorised subject terms or codes from a particular scheme, you are allowed to do so, and syntax will exist to support this. Furthermore, TYPEs should only be used where they clearly and significantly improve search precision. What is essential is that the definitions of all the elements are such that an unqualified value has a clear and useful meaning.

Specific structural proposals

A proposal for the use of qualifiers (both TYPEs and SCHEMEs) across the fifteen Dublin Core elements has been formulated by a group led by Rebecca Guenther of the Library of Congress. This proposal may be found at http://www.loc.gov/marc/ dcqualif.html (Dublin Core Qualifiers/ Substructure : a proposal. Last updated: 15 April 1997).

My view is that, of the fifteen Dublin Core elements, probably ten of them can (like SUBJECT) have unqualified free text as their default value, with a SCHEME being an optional addition. However, probably five of them cannot. In other words, something like five elements (again, in my view) require either a SCHEME or an authorised list of values as the default standard. One of these, RESOURCE IDENTIFIER, we have already discussed. The other four which probably require some structure in their default mode are DATE, RESOURCE TYPE, LANGUAGE and COVERAGE. My view is that the use of free text words in these five elements will fail to deliver satisfactory search precision. For example, a RESOURCE TYPE can be expressed in many different ways (article, paper, contribution, etc.) and without a controlled vocabulary the user will have to enter or guess all of the appropriate synonyms.

In June and July 1997 there was a flurry of discussion on the Dublin Core list concerning resource types. The result of this discussion has been a proposed list of authorised terms for resource type, developed by Roy Tennant at Berkeley. (This list can be found at http://sunsite.berkeley.edu/Metadata/types.html). The list is hierarchically structured, in three levels. These terms provide a controlled vocabulary which will improve precision by allowing users to confine a search to a particular resource type. It should be noted that this list does not have universal acceptance. Some observers have criticised it as "half baked", but I believe that it provides a reasonable starting point for further development.

A structured approach has also been developed for the COVERAGE element, which describes the spatial and temporal characteristics of the information resource. This is another element for which improved search precision clearly depends on a clear and well defined data structure, and also on knowing what the data represents. The proposed standard (which may be found at http://alexandria.sdc.ucsb.edu/public-documents/metadata/ dc_coverage.html) allows the spatial or temporal characteristics to be specified using either text or numbers (such as geographic coordinates and date-time).

Similarly, with DATE we have the twin problems of defining which date we are talking about (date of creation, date of last modification, etc.) and of avoiding ambiguity in the field content due to different means of representing dates. It seems to be generally accepted that DATE would be best recorded using the ISO 8601 standard. However, there has also been discussion about extending ISO 8601 to allow for the inclusion of time, dates prior to the year zero, and dates after the year 9999. At one stage in this debate, somebody made the ingenious but probably facetious suggestion that date-time be recorded as the number of seconds following the Big Bang!

Similarly, it seems to be generally accepted that LANGUAGE would be best recorded using the ISO 639 standard.

The registration of schemes

The fourth Dublin Core workshop endorsed the idea of a Dublin Core Registry which would, amongst other things, support the registration of schemes that are cited in any of the Dublin Core elements. The schemes would be documented on the World Wide Web, allowing creators and users of Dublin Core metadata to retrieve information about the scheme, and to retrieve lists of authorised values if necessary.

A registration process implies that only certain organisations would be permitted to register particular schemes, or to delegate authority to maintain these schemes.

Practical implementations

The Dublin Core metadata element set is beginning to emerge from the context of theoretical discussion and into the context of deployment in practical projects. An example of its use in Australia is the EdNA Project (http://www.edna.edu.au). Amongst other things, EdNA provides a 'one stop shop' directory to relevant resources for all sectors of education and training in Australia, and supports easier access to high quality resources available on the Internet. It provides a mechanism for authorised educational organisations to enter metadata in their Web pages which are automatically recognised by the EdNA software to keep entries in the EdNA Directory up to date. The EdNA Project has played an active role in the discussion and definition of standard values in the RESOURCE TYPE field.

A well known international example of the use of Dublin Core is the Nordic Metadata Project (http://linnea.helsinki.fi/meta/). This Project has developed a template which allows anyone to enter details of an information resource into a form, and to have Dublin Core metadata for that resource generated rapidly and accurately, using the latest syntactical rules. The project has also developed a Dublin Core search service, and Dublin Core to MARC conversion software.

Projects such as this remind us that widespread development of metadata will not occur unless tools are available to simplify and streamline the process of metadata creation and maintenance. An important new project in Australia, funded by the AVCC, aims to develop and disseminate such tools. The MetaWeb project (http://www.dstc.edu.au/RDU/ MetaWeb/) is a joint project of the Distributed Systems Technology Centre, the Australian Defence Force Academy, Charles Sturt University, and the National Library of Australia. The Project will develop these tools in the context of the repository model, in which metadata would be stored in a local repository rather than embedded within HTML pages using the META tag. The data in these local repositories would be gathered regularly by the harvesting engines.

Complexity versus simplicity

At the fourth Dublin Core workshop, librarians were in the vanguard of those who warned against successive complexity, against turning Dublin Core into another MARC. Responding to warnings of this kind, Peter Graham of Rutgers University said:

"The subtext of the 'anti-MARC' discussion is a desire to keep the Dublin Core simple. That's all well and good, but the desires of users are complex.... Those who want to keep the Dublin Core record 'simple' ... have an obligation to describe what the limits are on what the Dublin Core record should supply. Those (like me) who want to see the Dublin Core capable of expanding into considerable capability have an obligation to describe how this can be made possible while preserving the "core" concept".