International Newspapers Conference:Asia and the Pacific

Canadian developments for the digitization of newspapers

Sandra Burrows, Newspaper Specialist, Reference and Genealogy Division, Library and Archives Canada

[Note: this paper is additional to the conference program - Sandra Burrows was unable to attend the conference, but has submitted this paper for publication on the website for the information of site readers]

Abstract
Canadian researchers generally feel that of all of the wealth of material that could be on Canadian web sites, newspapers would be the easiest to digitize and search. Vast parts of newspaper collections, such as photography collections and birth, marriage and death notices have already been scanned and indexed. These parts have seldom been taken directly from the printed or the microfilm editions. Rather, genealogical societies and a consortium of contributors undertaking historical projects usually scan existing word documents and finding aids, in the case of genealogical information, and contact prints or scrapbooks in the case of newspaper photographs with added historical arrangement and context.

Holistic digitization is much more problematic. The questions of copyright, storage, access, editions, cost recovery are among a few that will be addressed at the end of this paper. Generally, Canadian newspaper digitization projects, in particular, those undertaken by Library and Archives Canada are thematic digitization projects. To be more specific, newspapers have been used to enhance and illustrate particular themes in the Canadian experience such as hockey, weather disasters or the opening of the west. This paper examines a few of these projects, as well as the present and future direction of digitization projects at Library and Archives Canada. It also presents a general overview of other projects across Canada and concludes with a review of issues in the digitization of newspapers.

Background
In October 2002, Library and Archives Canada convened a national consultation to examine the issue of a national strategy to strengthen access to Canadian newspapers online. In addition to discussions on "born-digital" newspapers and access to these, the consultation participants also considered the issue of digitization of Canadian newspapers already existing in other formats, primarily print and microfilm. Organized in association with the Canadian Initiative on Digital Libraries (CIDL), the Canadian Newspaper Association (CNA), and the Association for Canadian Studies (ACS), the consultation drew together experts in the fields of newspaper publishing, micro graphics, preservation, digitization, information dissemination, librarianship, archival science and scholarly and genealogical research. The consultation's outcomes included an impassioned plea for larger-scale full-text digitization of newspapers, a call for Library and Archives Canada to exercise leadership in ensuring long-term access to Canadian newspapers online and a multi-stakeholder approach to a national strategy. More details on this initiative are available in Making Known a National Collection of News Media in Canada: a report to the China Newspaper Seminar Tuesday, March 30, 2004, available at http://www.ifla.org/VII/s39/conf/Sandra1.pdf

National programmes
In the 1980's, Canada, Britain and the United States embarked on separate yet similar national programs for their respective newspaper heritage. There are variations according to country, yet the goals are the same: 1) to identify newspapers (titles and holdings); 2) to preserve them; and 3) to make them accessible. Twenty years later, all three countries have virtually met the first goal of identification and description. The second goal, preservation, requires a greater investment of time and resources. For newspapers, the accepted preservation medium is microfilming and preservation microfilming is continuing at varying levels of intensity.

In Canada, heritage institutions, consortia, commercial micropublishers and newspapers themselves do preservation microfilming. Library and Archives Canada acquires newspapers on microform partly through legal deposit for post 1988 papers for which more than four microfilm copies are produced and partly through purchase. The collection now numbers over 300,000 reels.

In conjunction with identification and preservation, all three countries turned to the third goal: making the content of newspapers accessible. When the national programs began in the 1980's, accessibility meant indexing selected content; in the current context, accessibility means digitization and full-text searching.(fn1)

At present, negotiations are underway to position Library and Archives Canada as a key player in making Canadian newspaper content accessible. As in many countries, legal deposit does not apply to digital content on the web. Library and Archives Canada, however, a "single, modern-knowledge institution" has extended legal deposit to online publications and to a sampling of Web sites. The Library and Archives Act and its related regulations, and a major initiative on the development, management and care of LAC's digital collection, will allow us to explore agreements with newspaper publishers to make known more newspaper content.(fn2)

Library and Archives Canada Web Sites with Canadian Newspapers
Several thematic projects on Library and Archives Canada website have significant newspaper components.
For finding aids, there is a Newspaper page that includes a list of Canadian newspapers on microfilm and a Checklist of Indexes to Canadian Newspapers. The thematic sites include:

Other Web Sites for Canadian Newspapers Online
There are other major web initiatives whereby Canadian newspapers are scanned and their content is available: Paper of Record, Pages of the Past (the Toronto star) and Globe and Mail: Canada's Heritage from 1844. Library and Archives Canada has site licences (licences based on the number of users an institution serves and the number of concurrent users) through their Intranet pages to all three of these products. An Ontario product, HALINET, although not solely a newspaper digitization source, provides a large number of digitized community newspapers on its local education pages.

All three major web products originated from Cold North Wind, Inc. which was established in 1999. Its first paper was the Melbourne Port Phillip Herald. In 2002, Cold North Wind, Inc. acquired the non-exclusive rights to scan and index the Canadian Library Association's microfilm, which covered titles from every province and territory from the late 1700's to 1930. Since then, some community newspapers' complete runs on microfilm have been added up to at least 2001. Titles are added frequently. Subscribers may access the product through their library's site licence or individually whereon they are charged for a monthly or yearly subscription. Paper of Record offers full-text and date searching and pages can be viewed, saved, printed locally, or high-quality prints ordered and sent by email to another site. Search words are located through an OCR process (which is also available for purchase by media companies) and highlighted. As with any digital product, the images on Paper of Record are only as good as the microfilmed product from which they are scanned. Not surprisingly, Paper of Record has encouraged preservation microfilming in order to ensure a high-quality product and further preservation of the original formats. Searchers are also able to contribute to the prioritization process by commenting on the newspapers or newspaper pages (as some scanning is incomplete) that they would like to see done. Paper of Record also has a featured paper, a featured historical subject with a number of scanned newspaper from a recognized expert source of news on it and a This Week in History Section. Paper of Record now includes over 500 newspaper titles from 16 countries comprising 8 million pages of newspaper images.(fn3)

The Toronto Star: Pages of the Past was the first digitized newspaper in the world (fn4). At a rate of one million pages per month, the paper has been digitized from 1894 now up to 2002 (two year, 1892-1894, are not available except for the first issue, November 3, 1892 as the originals were not available at the time of filming). The newspaper pages in Pages of the Past are high-resolution scans that average 700KB each. Subscriptions through the Torstar Syndication Services are available on 1, 24, and 72-hour bases as well as on 7 days to a one-month basis. The Toronto Star is the largest English-language newspaper in Canada with 2.2 million readers.

The Globe and Mail: Canada's Heritage from 1844 is the second-largest newspaper to be digitized in Canada. It runs from an earlier period in Canada's history, 1844 (20+ years prior to the confederation of Canada's four largest provinces) and goes to 2001.(fn5) It is now made available on a subscription basis through Micromedia ProQuest, which allows the product to reach the more lucrative Canadian Studies market in the United States and other countries. An added feature of this product is the ability to view the results in thumbnail format as well as full-page pdf's with the highlighted search word.

Micromedia has made available another option for both products; namely, perpetual access whereby, subject to license agreements, on a non-exclusive, non-transferable basis, subscribers may download one electronic copy of the database for storage on their internal system. For access (as of 2004) a site would pay a one-time fee of four times their current site license price or $10,000 (U.S.) whichever is greater. A site can choose to discontinue subscriptions and keep the file (in the case of Globe and Mail, perpetual access covers 1844 to 1950) and if they were to continue their subscription, the annual cost would be 65% of what they would have paid if they had not opted for perpetual access. It is also available for both the unlimited access and simultaneous user options and has been made available to consortia subscribers.(fn6)

Non-Commercial Web Sites
In a recent media release, the Bibliothèque nationale du Québec announced an intention to add to its web site more than three million pages of Quebec newspapers from the 19th century up to 1950 as well as cultural reviews from 1900 to 1950.(fn7) These materials will be available free-of-charge. Société canadienne du microfilm, inc. a company that has filmed a large number of historical French-language newspapers in Canada, has been under negotiations with La presse and other French-language newspapers as well as Olive Software to look at digitization of some of the historic pages.

Another Olive Software product available through the University of Alberta Libraries is a digitization of three Franco-Albertan newspapers, La survivance, Le franco-albertain and Le franco from 1928 to 1967. This is part of a larger Alberta Heritage Digitization Project which is an ongoing project to digitize historical information which is otherwise difficult to access including Alberta folklore; art; aerial photographs; maps; early Alberta newspapers; Alberta's retrospective bills, statutes, and gazettes; and local histories. This project has made use of collections from the University of Calgary, the Provincial Archives of Alberta, the Glenbow and Nickle Arts Museums, the Historical Society of Alberta, and the Universities of Alberta and Calgary and is funded by Canada's Digital Collections Initiative at Industry Canada, the Millennium Bureau of Canada, Alberta Historical Resources Foundation, the Alberta Knowledge Network, the Alberta Library, the University of Calgary, the Alberta Law Foundation, the Calgary Foundation, and the Calgary Community Lottery Board.(fn8)

The HALINET (the HALton Information NETwork) is a partnership of public libraries, the school boards and Sheridan College - all located in the Regional Municipality of Halton in Ontario. The project includes a unified Halton County set of local newspaper indexes from a variety of sources with the added feature of being able to search more than 65,000 newspaper images in black and white and in colour (the colour images are produced directly from the print edition; the black and white from microfilm editions). HALINET employs Lizard Tech's MrSID (Multi Resolution Seamless Image Database) for better delivery of the images over the Internet for purposes of printing. The image files of the newspapers are processed through MrSID and patrons are able to zoom in on the picture or article in which they are interested, and print just that. Alternatively, they are able to shrink the image of an individual page of a newspaper and print that.(fn9)

The University of Alberta Library and the Manitoba Library Consortium are collaborating on Manitoba: Life and Times, a digitization project that will include community newspapers and make them available free-of-charge in a full-text searchable online interface. (fn10)

The growth of digital newspaper and newspaper indexing pages that feature Canadian newspapers and Canadian newspaper indexes is almost unmanageable. In addition to the snapshot approach that Library and Archives hope to take of this area, we also maintain a list of newspaper web pages, indexes and non-Canadian pages which feature Canadian news sites at News Sites - Internet / Sites d'informations sur l'Internet at:
http://www.collectionscanada.ca/8/16/r16-211-e.html and
http://www.collectionscanada.ca/8/16/r16-211-f.html

Issues and Concerns
There are quite a few reasons why Canada, and perhaps many other countries have been slow to adopt either whole-scale funding or support for full-text newspaper digitization on a grand scale. The following are the difficulties and challenges that almost any institution has faced at one time or another concerning not only newspaper digitization and access but, in general, access to newspapers in any format as often the same frustrations apply.

  • Unlike other types of information, newspapers tend to be a conglomerate of styles, fonts, media, presentations and content. While handwritten documents tend to be the most challenging types of documents to scan and index, especially those with a variety of authors, newspapers can be equally challenging. The type of OCR required to not only interpret different kinds and size of print but also photographic images must be quite sophisticated and sophistication means higher cost
  • The digitized product is only as good as the source. All of the inherent problems with microfilm are magnified with the digital product. These include poor quality of the filming, missing microfilm masters, incomplete editions, removal of certain parts of the newspaper from the microfilm product and huge storage costs (although these are decreasing). As well, the cleanup; i.e. removing speckling, scratches, straightening images, and removing borders around pages, is usually dependent upon a proprietary software program.
  • The bottom line for newspapers is profit; the bottom line for publicly-supported libraries and archives is making information available free of charge to users while preserving original content. The goals of public and private sectors can be mutually exclusive. Most libraries must support digitization in conjunction with some preservation of the original, usually through the recognized standards that apply to preservation microfilming. Most newspapers are not particularly interested in microfilm other than as a revenue-generating part of their business. As libraries opt for digital products, they tend to decrease their subscriptions to print and microfilm editions of newspapers in exchange for the more accessible CD-ROM or online product and most libraries do not have the luxury of subscribing to both versions except perhaps for their local newspaper.
  • In Canada, most microfilming of newspapers is commercial yet less than 1/10th of the newspapers filmed in Canada are filmed in four or more copies. Thus, in Canada, 9/10th of newspapers filmed are not required to be sent on legal deposit to Library and Archives Canada. Without subscribers, filming decreases and even fewer libraries or local archives or historical societies maintain large print collections. Although Library and Archives Canada does maintain a large print newspaper collection, these papers are in bound volumes and would require refilming to be used in any major wholesale digitization project.
  • Digitization of coloured pages still requires the use of originals and supplements to print newspapers are often not included in subscription prices. Copyright protection for newspapers is rarely available for the whole newspaper and any subscriber to online systems can relate the frustration of finding whole or parts of stories removed as copyright permission was not obtainable or the copyright for a part of a paper is not available on that particular system. In addition to the copyright as it applies to newspapers as a whole, copyright varies widely as it applies to photographs, republished material, bylines, cartoons, letters to the editor, editorials to name a few.
  • Newspapers are still searching for a profitable business model for the web version of their product. They have tried putting up their newspaper archives for less than a year or requiring that subscribers to the Net version must be print subscribers. There has been a recent flurry of activity by some of the larger publishers to invest in wholly Net-based products whose audiences primarily access their information on line. In Canada with relatively few independent daily newspapers, the newspaper chains dictate the same digitization and indexing practices and, by and large, they have not marketed parts of their product as a Web source to any particular audience; e.g. their birth, marriage and death notices to genealogists.
  • For libraries and archives charged with preserving Canada's heritage, digitization poses an interesting dilemma. Most institutions have long given up the idea that they can or should preserve everything centrally but the attitude toward digitization as replacement technology still remains fairly strong especially at the management level or the government level deciding on budgets. Planning for long-term preservation needs to receive more attention. There are no proven standards for digital preservation except for migration of content to other formats and the Association of Research Libraries endorses digitization as an accepted preservation reformatting option. However, many institutions that digitized material and did not keep the originals are finding that they now must produce a microfilm edition from the digital product and some have even reproduced printed copies of digital documents in order to save the content. However, reproduction from the digital content means the loss of the original audio files and digital videos from the born-digital versions.(fn11)

Conclusion
The newspaper, as we know it, is changing. It flashes, it beeps, it interacts with other media, and it disappears to reappear in another version and reinvents itself several times a day. It is, as one standards expert puts it, a "moving target".(fn12) In planning digitization projects, we need a digital strategy and some tough decision making as to what can be preserved and in what format and perhaps most importantly, who is responsible and who will take responsibility in the long term.

At a recent Symposium on Web Archiving held at Library and Archives Canada February 3, 2005, Ingrid Mason stressed the need for institutions to demonstrate the integrity of what they do in building a digital archive.(fn13) In order to arrive at a common goal, it is difficult to meld the aims of institutions, which promote preservation and free access with those of the news business that is constantly experimenting to stay ahead commercially. Future Canadian digitization projects must reflect clear thinking and a common goal to preserve a part of Canada's heritage that is rapidly disappearing. Without a strategy or leaders to convince legislators and funding agencies and a mandate to facilitate access, large-scale co-operative projects will not be successful.

(fn1)The above information under Background is taken from the Draft Centre for Newspapers and News in Canada: a document for discussion by [Library and Archives Canada] Management Board. January 2005 by Mary Jane Starr, Director-General, Centre for Newspapers and News in Canada. (Ottawa: Library and Archives Canada, 2005), pages 2-3.(back to text)

(fn2)Performance Report for the period ending March 31, 2004
The National Library of Canada and National Archives of Canada Performance Report for the period ending March 31, 2004, was tabled in Parliament on Thursday, October 28, 2004. (back to text).

(fn3) About Paper of Record http://paperofrecord.com/Default.asp and Cold North Wind, Inc http://www.coldnorthwind.com/ Who We Are pages as well as notes from the lecture by Bob Huggins, President and CEO of Cold North Wind at: Canadian Newspapers Online: A National Consultation: toward a national strategy to strengthen access to Canadian Newspapers Online. Speaker Notes. Ottawa: National Library of Canada, 8 October 2002.(back to text)

(fn4)Notes from the lecture by Bob Huggins, President and CEO of Cold North Wind at: Canadian Newspapers Online: A National Consultation: toward a national strategy to strengthen access to Canadian Newspapers Online. Speaker Notes. Ottawa: National Library of Canada, 8 October 2002. (back to text)

(fn5)Ibid. (back to text)

(fn6)Message November 22, 2004 posted on concan-l on behalf of John Durno.
The Perpetual Access Addendum is available at: http://www.micromedia.ca/products_services/Perpetual_access.htm (back to text)

(fn7)La tribune. Sherbrooke, Quebec, February 18, 2005 p. D3 D'ici peu s'ajouteront plus de trois millions de pages de journaux québécois du XIXe siècle jusqu'en 1950 ainsi que plusieurs revues culturelles des années 1900 à 1950. (back to text)

(fn8)More information on this project is available at: http://www.ourfutureourpast.ca/about.htm (back to text)

(fn9)Information has been taken from the HALINET description on CIDL (Canadian Inventory of Digital Initiatives at: http://www.collectionscanada.ca/initiatives/; specifically at: http://www.hhpl.on.ca/halinet/demo_home.htm [note: this demo page appears to have disappeared early 2006]. (back to text)

(fn10)Web announcement Tuesday, March 1, 2005, Email sent on behalf of the School of Library & Information Studies, University of Alberta to slis-everyone@mailman.svr.ualberta.ca on the subject of a Research Seminar, Manitoba Metadata: Digitizing Manitoba's Cultural Heritage presented by Bess Sadler, to be held Monday, March 7, 2005 at the School of Library and Information Studies, University of Alberta, Edmonton, Alberta (back to text)

(fn11)Digital dark ages: drastic action needed to preserve computerized photos and other files. By Sarah McGinnis, Telegraph-Journal, December 11, 2004 New Brunswick Telegraph-Journal. p. A1/A8
"Preserving digital files may mean avoiding computers altogether. The provincial archives is putting the most vital information back on microfilm, which avoids the problem of "CD rot" and disintegrating data. They are also reverting back to paper by printing copies of digital documents." (back to text)

(fn12) NML: the News Business Gets Serious about Content. By Barry Schaeffer Newspapers and Technology Magazine, June 4, 1999. (Reprinted at: http://www.xsystems.com/nml.htm) (back to text)

(fn13)Ingrid Mason. Presentation on New Zealand Digital Web Archiving at Symposium on Web Archiving, February 3, 2005, Ottawa, Canada. (back to text)

About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us