Introduction: definition and scope
In 1998 the Digital Library Federation offered the following definition of 'digital libraries':
"Digital libraries are organizations that provide the resources, including the specialized staff, to select, structure, offer intellectual access to, interpret, distribute, preserve the integrity of, and ensure the persistence over time of collections of digital works so that they are readily and economically available for use by a defined community or set of communities".(Council on Library and Information Resources,1998)
This definition emphasises that 'a digital library, like any library, is more than a mere aggregation of information resources: it is a service which is based on principles of selection, acquisition, access, management and preservation, related to a specific client community' (Cathro, 1999). Thus any analysis of the economic aspects of digital libraries will need to take account of the costs of all of the above activities, while also examining issues of value and benefit. In the definition above, 'collections of digital works' can encompass a wide variety of manifestations, including digitised content created by libraries, museums and archives; digitised newspaper collections; web archives and other 'born digital' content; university repositories; and collections of digital research data.
Australian work on economic models
Little research has been undertaken in Australia on economic models for digital libraries. Most Australian commentators on this topic have been content to cite the models and findings from research undertaken outside Australia. For example, in his 'Sustainability issues paper' prepared for the Australian Partnership on Sustainable Repositories, Bradley cites the work undertaken by Brian Lavoie and his framework of 'responsibilities, incentives, and organization' (Lavoie, 2004). Bradley notes that digital library content requires persistent attention in order to preserve it:
"Benign neglect, with its lower cost profile, will not ensure preservation....The requirements of ongoing sustainability demand at their base a source of reliable funding, necessary to ensure that the constant, albeit potentially low level, support ... can be maintained for as long as it is required'. Such funding, he notes, is 'not at all typical of university based communities." (Bradley, 2005)
Other Australian commentators have cited economic models developed in the United Kingdom, including the Lifecycle Information for E-Literature (LIFE Project, 2006) and the Digital Preservation Coalition's Preservation Management of Digital Material Handbook (2006). This Handbook also deals with the business model for the digital library, including the issue of benefit and value. It references approaches, such as the British Library's use of 'contingent valuation', to 'enumerate the value of collections and services which had previously appeared to be unquantifiable'.
Some Australian commentators have also explored this challenging area. For example, Missingham (2005) has examined the use of 'contingent valuation' by the British Library and by some public library services in the United States, and has reviewed some value studies of national bibliographic services in Canada and New Zealand.
An important business case for digital library development is represented in a report prepared for three of Australia's major cultural institutions: the National Archives, the National Library and the National Film and Sound Archive. In 2006 those three institutions developed a joint proposal for additional funding to enable them to cope with the challenge of collecting, managing and preserving a rapidly increasing volume of digital content, while continuing to fulfil their responsibilities for print and other traditional content.
After this bid failed to attract policy support at its initial attempt, the three agencies engaged one of Australia's leading economic consulting companies (Access Economics) to conduct a cost-benefit analysis of the funding proposal. Access Economics developed a model which assessed the cost estimates prepared by the agencies against estimated benefits. This benefit model analysed the value of the proposal from the perspective of society as a whole. The consultants considered both an 'asset approach' and a 'user approach'. The asset approach looked at the stated value of traditional-format assets held by the institutions and developed a digital format equivalent value. The user approach looked at the costs and time incurred by users in accessing digital content compared to traditional content. Access Economics commented that both the asset and user approaches were likely to provide a relatively conservative benchmark for value, as neither takes account of any 'consumer surplus' (the willingness of some consumers to pay more than the market price).
The consultants commented that 'in general, we favour measurement under the user approach where there is a discrepancy, as the asset approach requires a conversion factor to equate stated values of traditional format assets to digital format assets (which have not been formally valued), and does not reflect the value of future usage.' The study examined costs and benefits over the long term, taking the period from 2008 to 2051 as the reference period. For this period, the bottom line was that benefits exceeded costs by a factor of six. The consultants commented: 'The benefits are significant and in large part accrue to users of the agency's material. The investments allow more material to be collected or preserved, and that material in digital form is more accessible to potential users than it otherwise would be' (Access Economics, 2007).
Digital libraries in Australia
Digital libraries in Australia are still at a relatively early stage of development. An overview of digitisation activities in Australia and New Zealand was given by Cathro in 2007 (Cathro, 2007). To date, the major achievements have included:
- digitisation of cultural heritage collections by institutions such as the National Library, National Archives, state libraries, public libraries and the Australian Institute of Aboriginal and Torres Strait Islander Studies;
- harvesting and archiving of the Australian web domain by the National Library and partner institutions;
- establishment of institutional repositories by the university community, usually with the active participation of university libraries;
- a strong record of collaboration, especially in projects funded through the national higher education portfolio.
This overview will examine these four aspects from an economic perspective.
Digitisation, the conversion of analogue information into digital information, is an important mechanism for building digital collections. The 'analogue information' may be instantiated in library materials such as books, pictures, maps, music scores, manuscript collections and audio or video files.
The process of digitisation encompasses all of the processes that are necessary to create usable digital files, including scanning or digital photography, metadata capture, creation of master and derivative digital files, and the transfer of these files into a suitable storage environment.
The National Library of Australia has been undertaking a routine collection digitisation programme for several years. This programme encompasses material such as pictures, rare maps, early Australian sheet music and selected manuscript collections. The Library has also been progressively digitising its collection of more than 40,000 hours of sound recordings. The Library has undertaken digitisation in-house, and its digitisation workflow is supported by a locally developed system known as Digital Collections Manager, which forms part of the Library's wider digital services architecture.
Internal National Library data indicates the following typical digitisation unit costs:
- about A$13 (£6) per picture;
- about A$37 (£17) for a music score containing five pages (as an average);
- about A$35 (£16) for a rare map;
- about A$17 (£8) for a manuscript item containing three pages (as an average).
These costs take account of the employee cost of scanning, quality control, creation and maintenance of metadata, creation of derivatives, uploading to the storage system and preservation treatment, as well as corporate overheads and the amortised cost of equipment.
These costs reflect the National Library's internal digitisation workflow and its own
quality procedures. The Australian Institute of Aboriginal and Torres Strait Islander Studies (AIATSIS) is conducting a major digitisation project as a result of A$12 million of funding by the Australian government in 2005. The project will enable AIATSIS to digitise up to a third of its film, video, photographic and audio collections. AIATSIS has reported digitisation costs of about A$14 per photograph and about A$25 per map or poster.
Most of Australia's state and territory libraries are undertaking digitisation programmes. The State Library of South Australia has reported digitisation costs ranging from about A$7 per image for single sheets of size A4 and smaller, ranging up to about A$33 for maps and other single sheets of size A0.
The National Library has also embarked on a project to digitise nine major Australian newspapers, covering out-of-copyright issues from 1803 to 1954. More than three million pages will be digitised. These pages will be made fully searchable and browsable through the use of optical character recognition (OCR) and other conversion services. This is a project with a strong collaborative aspect, with support by the state libraries in selection of newspaper titles and in the development of off-budget funding for digitisation of regional newspapers. The state libraries have supported a model involving the shared use of the National Library's infrastructure, including the workflow and delivery software, storage infrastructure and database platform. From a national perspective, this collaborative approach will create efficiencies in the digitisation and delivery processes.
Newspaper digitisation typically involves an 'industrial scale' process involving millions of pages, and it is common practice for major components of the process to be outsourced to companies which employ staff based in countries with low to moderate employee costs. Depending on the workflow, quality standards and degree of preservation treatment required, the unit cost of a multi-million page newspaper digitisation project could fall in the range £0.75 to £1.50 per page.
Australia was a pioneer in the field of web archiving. Since 1996 the National Library of Australia has been developing and maintaining PANDORA,1 an archive of selected significant Australian websites and web-based online publications. The purpose of PANDORA is to ensure that Australians of the future will be able to access a significant component of today's Australian web-based information resources (Cathro et al., 2001). PANDORA is now a routine part of the selection, acquisition and collection management processes of the Library. PANDORA is a collaborative activity, as the archive is being built by the state libraries and some other cultural institutions in addition to the National Library. This collaboration involves a shared software and database platform and agreement on non-overlapping collection responsibilities. This collaborative activity improves the productivity of the archiving and content delivery process.
Nevertheless, it is clear that the selective model, as exemplified by PANDORA, has relatively high unit costs. The model involves individual selection of suitable websites, a dialogue with the website publisher to obtain permission to archive and republish, determination of a gathering schedule, cataloguing of the website at the title level, quality assurance checking of the gathered content, and (if necessary) actions to deal with any quality problems.
Phillips has presented the results of a study of acquisition costs for the PANDORA Archive (Phillips, 2005). This study found that, for the National Library of Australia's workflow, the cost of acquiring each instance of a selectively archived website was A$179 (at current exchange rates (April, 2008), this is equivalent to £81). It should be noted that the selective approach gives the Library a legal basis for opening the PANDORA Archive to public access, and that the creation of title-level metadata can assist in integrating the discovery of websites with the discovery of other collection resources.
Since 2005, the National Library has also been undertaking an annual large-scale harvest of the Australian web domain (Gatenby, 2006). This activity was the result of a long-standing aspiration to complement the selective PANDORA approach with a whole-domain approach. The Library contracted the Internet Archive2 to undertake the whole domain harvest.
In each of the most recent two annual harvests, the Library captured more than 500 million files, comprising over 19 terabytes of data. Each of these harvests represented more than seven times as much data as was gathered during the first nine years of harvesting for the PANDORA Archive.
The cost of the annual Australian Whole-Domain Harvest is about A$260,000: this cost includes the cost of the crawling process, the storage infrastructure and the indexes. As each harvest contains about 1.2 m website hosts, the cost of acquiring each host instance is about A$0.22.
Clearly, then, the cost of selective archiving, using the National Library's workflow, is several hundred times that of the whole-domain approach. This is to be expected, given the labour intensive processes of selection, cataloguing and quality assurance associated with the selective approach.
In recent years in Australia, as in many other countries, we have seen the rapid development of repositories to assist universities to manage and expose the research outputs of the academic community. Before 2004, a few Australian universities had been using the Southampton E-prints software or DSpace as platforms to build these repositories.
From 2004 a number of projects funded by the National Higher Education Ministry (now called the Department of Innovation, Industry, Science and Research) have stimulated widespread development of repositories (Cathro, 2006). The key projects have been:
- the ARROW Project (Australian Research Repositories Online to the World), led by Monash University;
- the Australian Partnership for Sustainable Repositories (APSR), led by the Australian National University (ANU);
- the MAMS (Meta Access Management System) Project, led by Macquarie University; and
- the Australian Scheme for Higher Education Repositories (ASHER), which provided A$25.5 million over three years to support the installation or upgrade of university digital repositories or their technical and administrative support.
The decision by the National Higher Education Ministry to invest in the establishment of institutional repositories has had a significant impact. By early 2008 more than 70 per cent of Australian universities had established repositories and collectively more than 90,000 digital collection items were accessible through these repositories. The range of software platforms had expanded to include VITAL (provided by VTLS Inc., based on the open source Fedora software, with funding support from the ARROW Project), Fez (developed by the University of Queensland, also based on Fedora, with funding support from the APSR Project) and ProQuest's Digital Commons.
Within the APSR Project, the National Library and ANU collaborated on the development of an Automatic Obsolescence Notification System (AONS), in the form of an open-source, platform-independent tool that can be used by repository managers (Pearson, 2007). The development of this tool was completed in October 2007. In the long term, this tool, and the collaborative activity which developed it, will improve the productivity of repository managers in dealing with format obsolescence, and thus help to reduce some of the costs of the 'persistent attention' that is needed to ensure digital preservation.
These investments have been mirrored by government action to manage and share research data. This issue was fostered through the work of the Prime Minister's Science, Engineering and Innovation Council and its report on Data for Science (2006). It has now been given concrete form through the establishment of the Australian National Data Service (ANDS). It is hoped that by late 2008 ANDS will be developing towards a coherent national programme for managing and accessing digital research outputs.
The ARROW Project has led the development of a national discovery service, now containing more than 90,000 records, which provides a single access point to the university repositories. This service will evolve into a broader discovery service under the auspices of ANDS and will need to cater for authenticated discovery of closed-access publications and research data. In doing so, it will leverage off another major Australian project aimed at developing improved access control services, namely the MAMS Project.
MAMS has developed a federated authentication model and software tools using the Shibboleth approach. A network of identity providers and service providers will participate in a national 'Access Control Federation' which will be established as a by-product of the Project. The benefit of this approach is that service providers will not need to set up and maintain their own user directory and authentication systems.
Collaboration and standards
Collaborative activities can influence the economics of digital libraries in several ways. Development of shared infrastructure and shareable software can assist in reducing the costs of developing and maintaining digital library services. Collaborative activities can also increase the value of digital libraries from the user perspective.
Australia has been notable for a high level of commitment to collaborative activities, often on a national scale. We have already seen examples in collaborative newspaper digitisation and web archiving, and in the collaborative projects resulting from investment in research
One important area of collaboration is the sharing of software components through the open-source model. A number of digital library developers have recognised the value of implementing a service-oriented architecture and initiatives to develop service frameworks have been established such as the E-Framework for Education and Research. The National Library of Australia is also working in this space, and has developed its own digital library service framework. The Library has announced its intention of working with other interested parties to develop further this framework as a collaborative activity (National Library of Australia, 2008).
One way of improving the value of digital libraries from the user perspective is to develop discovery and delivery workflows which provide the user with a coherent experience even when the content is widely distributed among many digital libraries. Currently, each digital library uses its own delivery interface. By developing a standard package for the exchange and delivery of digital library content, the user experience can be improved. Within the APSR Project, the National Library and the ANU collaborated on the development of the Australian METS (Metadata Exchange and Transmission Standard) Profile, an open and extensible standard for exchanging objects between repositories and for supporting both submission and dissemination workflows. In December 2007 this Profile was registered with the Library of Congress. The Profile has been described in a recent article (Pearce et al., 2008). In addition to improving the user experience, this Profile has the potential to improve the productivity of repository management by standardising workflows for the exchange of data between repositories, or the transfer of content if the repository platform is being replaced.
What are the issues confronting the further development of digital libraries in Australia? Perhaps the most challenging issues are those confronting Australia's cultural institutions. For them, the key issue is that there has been no coherent government response to the challenge of collecting in a digital world. There has been no recognition that such institutions face significant funding challenges in managing both traditional and digital collections. There has been no formal recognition by government of the digital preservation problem. At the national level, legal deposit has not been extended to cover digital collecting, despite a decade of lobbying for this change. No funding process has been established to support digitisation activities.
Australia's cultural institutions (archives, galleries, libraries, museums) have formed a loose alliance through the Collections Council of Australia. The Council is supported by the Cultural Ministers Council, a joint national/state government body. Following a national summit on Digital Collections in August 2006, the Collections Council issued an Australian Framework for Digital Heritage Collections which called, among other things, for a national policy and development plan for digital heritage collections, improved legal deposit legislation, and
enhanced funding support for digitisation and preservation (Collections Council of Australia, 2007). To date, the only action by government has been to release a discussion paper on the extension of legal deposit.
Individual institutions, such as the State Library of Victoria and AIATSIS, have received support for digital collecting activities. The State of Victoria, in its 2006/07 budget, provided A$25.1 m to the State Library of Victoria over four years to support a shift to a new service model based on online services. And, as we have seen, the federal government has supported digitisation and digital preservation of the collections of AIATSIS, including substantial parts of the pictorial, audio and video collections. The challenge remains for cultural institutions to
persuade governments to address the digital collecting imperative in a systematic fashion. Analyses such as that undertaken by Access Economics may assist in developing appropriate business cases.
Australia is notable for its record of collaboration and for its efforts to establish coherent national programmes. The most obvious signs of collaboration lie in the joint efforts of the national and state libraries (particularly in web archiving and newspaper digitisation) and in the development of national research infrastructure programmes. As we have seen, the picture is more encouraging in the higher education sector, given the explicit recognition and funding of university repositories by the federal Department of Innovation, Industry, Science and Research. While in other countries such developments are often dependent on shortterm project funding, in Australia this support promises to be placed on a recurrent footing through the establishment of the ANDS, which will provide assistance and tools to repository managers and individual researchers in the management of research outputs.
Digital preservation remains a very challenging issue for digital libraries, but in recent years greater optimism has been generated as a result of experience with web archiving, the development of tools such as AONS (Automated Obsolescence Notification System), and through work on standards such as the Australian METS Profile.
Access Economics Pty Ltd (2007) Cost-Benefit Analysis of Digital Funding
Proposals: Report by Access Economics for National Cultural Institutions, 27
August 2007. Access Economics.
Bradley, K. (2005) APSR Sustainability Issues Discussion Paper. Australian
Partnership for Sustainable Repositories, January.
Cathro, W. (1999) Digital Libraries: A National Library Perspective. Information Online & On Disc Conference, Sydney, 19–21 January.
Cathro, W. (2006) 'The role of a national library in supporting research information infrastructure', IFLA Journal, 32 (4): 333–9.
Cathro, W. (2007) 'Digitization in Australasia', Serials, 20 (1): 9–15.
Cathro, W., Webb, C. and Whiting, J. (2001) Archiving the Web: The PANDORA Archive at the National Library of Australia. 'Preserving the Present for the Future' Web Archiving Conference, Copenhagen, 18–19 June.
Council on Library and Information Resources (1998) CLIR Issues, 4 (July/August).
Gatenby, P. (2006) Recent Developments in Digital Archiving and Preservation. Paper prepared for the CDNL meeting, Seoul, August 2006.
Lavoie, B. (2004) 'Of mice and memory: economically sustainable preservation for the twenty-first century', Access in the Future Tense, CLIR Reports
LIFE Project (2006) Bringing Digital Preservation to LIFE: Full Report from the LIFE Project.
Missingham, R. (2005) Libraries and Economic Value: A Review of Recent Studies.
National Library of Australia (2008) Library Labs wiki.
Pearce, J., Pearson, D., Williams, M. and Yeadon, S. (2008) 'The Australian METS Profile: a journey about metadata', DLib Magazine, 14 (3/4).
Pearson, D. (2007) 'AONS II: continuing the trend towards preservation software "Nirvana"', iPres2007 (Beijing), 11–12 October.
Prime Minister's Science, Engineering and Innovation Council (2006) From Data to Wisdom: Pathways to Successful Data Management for Australian Science. Report to PMSEIC [by the] Working Group on Data for Science, December.