Selection of Newspapers for Digitization and Preservation: A User PerspectiveRoss Harvey Abstract Users of newspapers in library collections, or to which access is provided by libraries, are increasingly provided with choices of versions for use. Each choice carries its own implications for usability: using the original paper copy is likely to restrict the user to a physical location; microfilms are notoriously user-unfriendly, but they are portable; and web-delivered digitized versions can be accessed from anywhere potentially and, in some cases, allow text to be searched, but there may be bandwidth and other technological issues. The increasing digitization of newspapers raises many issues that libraries need to address. These include:
This paper addresses these issues from a user perspective, focusing on Australian and New Zealand experience. It describes recent research into a New Zealand newspaper, the Inangahua Times (Reefton) of which both the microfilm and the online digitized version was used. It notes some of the issues surrounding the use and usability of newspapers in hard copy, microfilm and digital forms. It provides examples from library practice and from user experience. Introduction: traditional selection practices This paper addresses some of the issues in making decisions about which newspapers libraries select to digitize and/or preserve. It provides a perspective from the point of view of the user, and focuses primarily on Australian and New Zealand practice. The selection of materials for preservation in heritage collections is traditionally made on the basis of certain characteristics of the materials. Much traditional preservation decision-making is driven by the physical condition of artifacts (Gertz 2000: 98). We can characterize the approach traditionally taken by libraries to selecting material for preservation as being based largely on preservation of items (or artifacts) in their original formats. Selection of items is based on key criteria, usually stated as age, evidential value, aesthetic value, scarcity, associational value, market value, and exhibition value, as in the U.S. CLIR (Council on Library and Information Resources) statement about the value of artifacts for research purposes:
Additional criteria are often applied, although these are not consistently recognized by all commentators as valid: physical condition, resources available, use, capital value, and an ill-defined, but nonetheless well-represented, category of social significance ('is the item held in community esteem'?). To these criteria we need to add user demand. This is noted in Harris's comprehensive summary of selection for preservation in the library context (Harris 2000: 206-224). She suggests the criterion 'Has the item been heavily used?', given that current use is posited as an indicator, albeit imperfect, of future use. These criteria apply to 'traditional' (that is, paper-based) materials, of which newspapers are, of course, an example. They need to be redefined (or, at least, refined) in an increasingly digital environment, an environment that includes newspapers, more and more of which are also available in digital form. This paper looks at some of the reasons why they need to be re-defined. Its perspective is that of a user of newspapers - in my case, as a researcher into the history of New Zealand and Australian newspapers. Consequently I present a view untrammelled by resource considerations or by any of the numerous and difficult political, financial, and other administrative issues faced by those who manage newspaper collections. User expectations: how researchers use newspapers Let me describe my most recent uses of newspapers for research purposes. I recently completed an article for a collection about the relationship between newspapers published in the British colonies and the British Empire's centre in London. In my contribution I examined, in detail, aspects of the New Zealand Press Association's dealings with the proprietor of the Inangahua Times. This newspaper was published in the remote mining town of Reefton, in the north-west of the South Island of New Zealand, and about as far away from London as one can get. This research required that I examine copies of the Inangahua Times from the late 1880s and the 1890s. I examined these copies, not from New Zealand, but from my current home base in Wagga Wagga, about three hours drive from Canberra. This was feasible for two reasons. The first was that a digital version of this newspaper is available from the National Library of New Zealand's web site Papers Past which provides 750,000 page images from New Zealand serials. The second is that I have in my possession microfilm of this newspaper produced by the National Library of New Zealand. It should be noted that, even if I was in New Zealand, it is doubtful whether I could get access to paper copies of this newspaper. The Inangahua Times cannot be considered to be a major newspaper, even in its brief heyday, and copies of it are rare. The set that was used to produce the microfilm and, later, the digital page images, was the only set approaching anywhere near completeness, and came from the Blacks Point Museum, located in a tiny settlement outside remote Reefton. When I saw them in 1986 these newspapers were housed in inadequate storage conditions and would have deteriorated under such conditions. (They are now in the custody of the National Library of New Zealand and, therefore, in high quality storage.) Using the digital page images as well as the microfilms of the Inangahua Times provided me with the opportunity to reflect on the advantages and disadvantages of both formats for users, and to these I have added comments based on previous experience in the use of fragile paper copies of New Zealand newspapers from the 1890s.
This list could, of course, be extended considerably; none of the above comments is likely to surprise members of this audience, or researchers who use newspapers in their investigations. However, a new factor has entered the equation, one that was not available until recently but with significant implications for the selection of formats for preservation and access of historical newspapers. I refer to the possibility of full-text searching of the text of newspapers. Increasingly, users of historical materials are being provided with the ability to carry out searches on the contents of these materials, with the possibility of doing this from their desktop. For newspapers the bar has been raised by offerings such as The Times Digital Archive and ProQuest's Historical Newspapers. These provide page images plus the ability to search the text. The Times Digital Archive can be searched in various ways: a keyword search (on words in the text, including advertising, and the metadata), a relevance search (a keyword search which weights the words according to their frequency), Boolean searching, and browsing by data (Readings & Holland 2003: 40-41; http://www.galegroup.com/Times/). ProQuest Historical Newspapers provides the ability to search the full text of selected U.S.-published newspapers by keyword, Boolean operators, and advanced search features (http://www.proquest.com/products/pd-product-HistNews.shtml). Software such as OCLC's Olive software is making it possible, for those institutions that can afford it, to provide searchable full text more readily. While I was preparing this paper I came across the most recent issue of RLG DigiNews [Feb 2005 issue]. It contains a detailed description of a project to digitize a significant Alaskan newspaper, the Tundra Times. The authors suggest that the approach described make it 'both technically and economically feasible for smaller organizations to undertake' digitization of historical newspapers. If I read this article correctly, they have made the full text of seventeen years of the paper (27,000 pages) and Internet-accessible for the sum of US$150,000 (Terpstra et al 2005). While this is still a large amount of money for many organizations, it does perhaps suggest that the costs of and techniques for digitizing and providing full-text access are now within the means and capability of many organizations, not just the large national and state libraries. Margie Barram's presentation about the Nambour Chronicle at this conference certainly indicates the feasibility for small libraries. What are the implications of all of this for selecting of newspapers for preservation? We have already seen that user demand is one factor in the selection of 'traditional' materials for preservation, and I suggest that user preferences will assume greater significance when we consider digitizing. And not only is it user preferences: user expectations will also drive the process. We are increasingly seeing how one-search, instant-result actions in information seeking are raising the expectations users have of information systems - a phenomenon which could perhaps be labelled 'the Google-isation of society'. I have written elsewhere that
People increasingly expect Internet access to their information sources, with easy-to-use search capabilities. Consider this quote:
We should not expect newspaper users to be any different. We need, therefore, to address the question of the extent to which user expectations should be accommodated in the selection of newspapers for digitization and preservation. Managers of newspaper collections have, then, a new set of issues to consider when thinking about selection. There are more and more choices to be made.
Each of these questions is examined in more detail. 1. Can digitizing be considered as a preservation mechanism, or is it only as a way to improve access? This is a complex question that has been heavily debated, so far without a universally accepted outcome. However, my recent study of digital preservation has led me to conclude that we now have sufficient experience of preserving digital materials to consider digitization of newspapers as a viable preservation option. This is also the conclusion reached by the Association of Research Libraries (Arthur et al 2004). There is, perhaps, an analogy with preservation microfilming. If certain standards are met and maintained (file format and metadata standards, for instance), if storage conditions meet specified requirements (through trusted digital repositories), and so on, then there is the very strong likelihood that the digitized files will be maintained in usable form into the future. And as with microfilming, it is probably prudent to keep the originals wherever feasible as a safeguard. (I do, of course, appreciate that the extremely poor physical condition of some newspapers means that they may not be in a state worth retaining after they have been handled for digitizing.) It should be noted that I am here not considering whether to digitize newspapers directly from the originals, or from high quality microfilm. I am not a subscriber to the Nicholson Baker school of thought and do not believe that the preservation of newspapers in their original paper copies is a realistic long-term option. The only realistic choices currently available for the long-term preservation of newspapers are microfilming to preservation standards, and digitizing. I have not looked at any comparisons of the costs of microfilming and digitizing for newspapers (if indeed any exist). Such a costing would need to encompass not only the production costs, but the long-term costs of storage, for example, of master negatives in cold stores, of digital materials in trusted repositories, and many other factors besides. Deegan and Steinvel suggested in 2002 that 'the cost-effective creation of usable and searchable digital content that offers users a realistic experience of the richness of newspapers is perhaps closer than we have hitherto thought' (Deegan & Steinvel 2002). At present all we can do is to assume that the costs are similar, or will become similar in the near future. Accepting this assumption, then, we can act on the basis of another: that user expectations will play an increasingly large part in preservation decision-making. For newspapers, this means that we need to provide digital versions with searchable text. 2. Do traditional selection criteria apply to selecting newspapers to be digitized? I want to step out of the role of user for a moment to note some of the technical and resource issues that we have to consider when thinking about selecting newspaper to digitize. Most of the published literature about selecting digital materials is about the selection of analog material to digitize, with the primary reason of improving access to that material. Digitizing programs are usually access-driven, with preservation as a by-product (Gertz 1999). The criteria identified for selecting materials for digitizing can, despite their emphasis on access, provide useful advice for selection criteria for preservation. Many of the traditional selection criteria apply, although the emphasis is different. For example, greater prominence is given to user demand and intellectual property rights (Gertz 2000: 98-99). Gertz has identified the criteria most frequently cited as:
We can add further to this by looking at other statements that articulate additional criteria. One is found in the Cedars Project Team report (Cedars Project Team 2002) which suggests that selection decisions for digital preservation must 'be pragmatic' and need to be based on the 'estimated value of the material, the cost of storage and support mechanisms, and the production of metadata to support the material' (Cedars Project Team 2002: 53). Specific primary criteria are proposed:
In similar vein, the Digital Preservation Coalition's Decision Tree for Selection of Digital Materials for Long-term Retention (Jones & Beagrie 2001: section 4) suggests that we consider selection policy (is there an institutional selection policy? Does the material fit into it? Is the material of significant long term value?), legal and intellectual property issues (have acceptable rights been negotiated? Can they be?), technical questions (can you handle the file format, now and in the future? Can the material be transferred to a more manageable format?), and the existence of documentation and metadata (has sufficient been supplied?). It is clear, therefore, that changes are required to the selection procedures developed for non-digital artifacts to accommodate digital materials. To the traditional factors new factors are added. The digital selection frameworks developed to date still place high priority on criteria for determining value, but also strongly emphasise criteria such as the legal and intellectual property rights governing a resource, whether we have the technical ability to preserve it, the costs involved in preserving it, and the presence of appropriate documentation and metadata. The table summarizes these factors.
We now need to consider such factors when we make selection decisions about newspapers. 3. To what extent should and can the expectations of users be accommodated as part of the process of selecting newspapers for preservation? In their broadest sense libraries exist to serve society, so of course users must be kept uppermost in mind in selection process. But it is users present and future who must benefit from these selection decisions and actions, and their requirements, as far as they might be identified, are not necessarily the same. Therefore, the immediate expectations and demands of present users should not be the paramount considerations. An anecdote illustrates some of the difficulties. In a much earlier career as the Newspaper Librarian for the National Library of New Zealand, I had input into the selection of newspapers for its preservation microfilming program. There was very strong lobbying from libraries and interest groups in Christchurch to make the Lyttelton Times, an important daily newspaper published in Christchurch from 1851 to 1929, a top priority for microfilming. Their experience was of this significant historical resource being so heavily used that physical damage was occurring. However, I could not give this newspaper high priority because, in the national scheme of things, it was not rare or endangered; there were are least three complete sets in the country, two of which were in excellent condition. User expectations must not be allowed, as this example suggests, to override all other criteria. 4. To what extent should and can the expectations of users be accommodated in the provision of digitized versions? The answer to this question is not so clear-cut. There is, I have suggested, an already strong expectation - one that is likely to strengthen further in the future - of the availability of searchable text versions of digitized newspapers. However, the user, if faced with the choice between access to digitized page images and having no digitized versions at all (that is, having access only to a microfilm version in the long term), is probably likely to opt for a 'no-frills' digitized version. There is also an increasing expectation that digitized materials should be available over the Internet. Newspapers are no exception. This is an expectation that I see no reasonable arguments for disregarding. It is now within the technical and resource abilities of most institutions to do this. Conclusion Users make up society, and societal values and imperatives ultimately determine what value is placed on heritage institutions such as libraries; the needs and expectations of users, therefore, are the basis of support that society is willing to provide to heritage institutions. In a crude sense, if users do not believe that heritage institutions serve their needs, politicians will not fund these institutions adequately. We need to go some way to meeting user expectations. The challenge is balancing user needs with what professional investigation has informed us are the needs of the materials we seek to preserve. Selection is about the nexus between user needs and material needs. When we select, we need to be aware of shifts in attitude by users on the one hand, and changes in the technologies that we have available on the other. The trick is to balance all of the competing factors so that the greatest number of interests and demands are satisfied. It is impossible to satisfy them all, of course, and the best we can hope for is to satisfy most of the people most of the time. References Sections of this paper are based on Harvey 2004 and Harvey 2005, which should be referred to for more detail. Arthur, Kathleen et al. (2004). Recognizing Digitization as a Preservation Reformatting Method. Prepared for the ARL Preservation of Research Library Materials Committee. http://www.arl.org/preserv/digit_final.html, (viewed 9 Feb 2007). Cedars Project Team (2002). The Cedars Project Report, April 1998-March 2001. http://www.leeds.ac.uk/cedars/OurPublications/cedarsrepmar01exec.html CLIR (Council on Library and Information Resources). Task Force on the Artifact in Library Collections (2001). The Evidence in Hand: Report of the Task Force on the Artifact in Library Collections. Washington, D.C.: CLIR. Deegan, Marilyn & Steinvel, Emil (2002). Digitizing Historic Newspapers: Progress and Prospects. RLG DigiNews 6(4). Gertz, J. (1999). Selection Guidelines for Preservation, in Joint RLG and NPO Preservation Conference: Guidelines for Digital Imaging. http://www.rlg.org/preserv/joint/gertz.html Gertz, J. (2000). Selection for Preservation in the Digital Age. Library Resources & Technical Services 44, 2: 97-104. Harris, C. (2000). Selection for Preservation, pp. 206-224 in Preservation: Issues and Planning (edited by P.N. Banks and R. Pilette). Chicago: American Library Association. Harvey, Ross (2004). 'Preserving Digital Documentary Heritage in Libraries: What Do We Select?', Symposium 2003: Preservation of Electronic Records: New Knowledge and Decision-making, Ottawa, Canada, 15-18 September 2003. Harvey, Ross (forthcoming 2005). Preserving Digital Materials: An Australian Case Study. München, K.G. Saur. Jones, Maggie & Beagrie, Neil (2001). Preservation Management of Digital Materials: A Handbook. London: British Library. Readings, Reg & Holland, Mark (2003). The Times Digital Archive 1785-1985: 'The Thunderer' on the Web. Update 2(7): 38-41. Terpstra, Judith A.K., Zarndt, Frederick, Ongley, David & Bodie, Stefan (2005). The Tundra Times Newspaper Digitization Project. RLG DigiNews 9(1). Biographical note Dr Ross Harvey is the inaugural Professor of Library and Information Management at Charles Sturt University's School of Information Studies, taking up his appointment in 1999. He has held academic positions in Australia, Singapore and New Zealand and has been a Visiting Professor at the University of California Los Angeles. From 1986-1987 he held the position of Newspaper Librarian at the Alexander Turnbull Library, National Library of New Zealand, and compiled the Union List of Newspapers Preserved in Libraries, Newspaper Offices, Local Authority Offices and Museums in New Zealand (Wellington: National Library of New Zealand, 1987). He has published extensively on newspaper history (focusing on newspapers produced in New Zealand during the nineteenth century) and on the preservation of documentary heritage materials. His current research is investigating the preservation of digital documentary heritage materials. Email: rossharvey@csu.edu.au |
| About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us |