International Newspapers Conference:Asia and the Pacific

Time's pivotal point - preserving the past for the future - NOW

Heather Brown and Andy Fenton

Introduction
We are at this conference precisely because newspapers contain unique information and knowledge.

Within a short space many of us will have moved from our current positions; over more time our collecting institutions will change; governments will come and go; political boundaries will change; funding will wax and wane. Ultimately one of the few things that will last will be the knowledge and the information that we preserve for future generations.

This knowledge is part of national and regional cultural heritage. Ultimately it is part of the cultural memory of the whole world.

In newspapers there is a unique cultural synthesis of the so-called 'big picture issues' and the everyday stuff of life - what Indian author Arundhati Roy has evocatively described as 'The God of small things' (Roy 1997). From regional and national perspectives on international events, to the opinions expressed in 'letters to the editor', to advertisements of wares for sale, radio and television programs, to the so-called 'hatch, match and dispatch' classifieds, newspapers provide a unique sociological perspective and commentary; a graphic insight into history, that continues to fascinate historians, sociologists, genealogists, fashion designers, students and the 'general public,' decades after they are first published.

It is no coincidence that the State library of South Australia chose the prominent head of a newspaper empire to officially open its building. One of the more popular exhibitions simply contains images taken from South Australian newspapers over time, counter-pointed against a timeline of significant cultural and social events.

Of course, we cannot preserve everything. Preservation is about making choices and Ross Harvey has covered key issues relating to selection for preservation.

Preservation as a risk management issue
The major theme of our paper is that preservation of newspapers is also a risk management issue. (Adams, Hart, 2004) Right now, we are standing at a pivotal point. On one side is the imperative to provide access to this information, both now and for the future, while on the other is the imperative to preserve in order to provide the access. There is a symbiotic relationship between the two.

Our theme is that a risk management perspective is a useful tool in maintaining a sustainable balance between these two imperatives. In this context we are talking about long term preservation risks.

There are a number of significant risk factors that have major consequences for the long term preservation of newspaper heritage.

Among these long term key risks are:

  1. Inherent nature of the materials
    • hard copy newspapers are produced on 'poor' quality paper which will deteriorate and become brittle over time due to the lignin content
    • digital repositories of newspapers need to be systematically backed up, refreshed and/or migrated to ensure their ongoing accessibility
  2. Quality. Surrogate copies of these newspapers, principally in microfilm but also now increasingly in digital formats, have not always been created to quality standards.
  3. Storage. As Henry Snyder has vividly described, the physical storage of the originals and microfilm copies has not always been managed to ensure their long term access. This also includes issues relating to poorly processed microfilms and the storage of older acetate films. More recently the risks relate to the storage of digital copies. We are aware of at least one Australian newspaper agency that has forever lost a portion of its digital newspaper database during the 1990s. This story would not be unique in the region.
  4. Bibliographic tracking of originals and surrogates. As the IFLA-PAC newspaper programs reflect, the systematic listing of newspapers and surrogate copies - microform and digital format - is critical to managing their long-term access and preservation. This role has traditionally been the province of national and state libraries and has been undertaken to varying degrees of success to date. While we are not cataloguers or database managers, we are familiar with the complexity of tracking newspapers that have common titles, different editions and frequent title changes, and undergo mergers and splits. Without clear national and regional union lists of the newspapers and holdings in all their formats, there will be the risk of duplication and at the worst - the potential loss of cultural heritage.

These risks are not new. Like motifs, they run through most of the literature about newspapers, microfilming and digitising over the past 15 years.

One of our key themes is that these key risk factors will need to be addressed as part of any broad regional strategies to preserve and provide access to newspapers in the future.

Risk management and microfilm
In our respective roles as a preservation manager in a state library and as a director of a preservation microfilming and digitising company, we both strongly advocate the advantages of preservation microfilm as a prudent risk management strategy for the 'now'.

In this context we are referring to preservation microfilm that is produced to well-established, rigorous international quality assurance standards. The IFLA Guidelines for Newspaper Preservation Microfilming and Supplement: Microfilming for Digitisation and Optical Character Recognition are based on such standards (IFLA 1996, 2002). Most of the people in this audience will be aware of the long life expectancy of preservation microfilm. Simulation tests demonstrate that microfilm produced to these standards has an estimated life expectancy of up to 500 years if it is stored according to the recommended environmental conditions. While many countries in the region aim to keep the originals for as long as possible, this will inevitably become more difficult over time. In a significant number of cases, preservation microfilm will last long after the originals have crumbled to dust.

In 500 year's time all that will be needed to read the newspapers on microfilm is a simple magnifying glass.

In comparison digital information is transient, and currently at least, requires frequent intervention to ensure accessibility in relation to rapid changes in software and hardware. In 500 years' time the likelihood of being able to access today's digital files will be extremely limited unless these files have been continually migrated/refreshed/emulated.

As Ross Harvey has highlighted the issue of whether preservation can be considered as a preservation mechanism is heavily debated. The conclusions reached by the recent ARL paper on Recognizing Digitization as a Preservation Reformatting Method (Arthur 2004) have been hotly debated within the international preservation community. Indeed the very same report states that standards and guidelines for long term preservation are still in the developmental stage, that issues of technological obsolescence need to be addressed and that management and storage costs are not fully known (Adams 2004).

'…indeed we expect digital formats will be very unforgiving of neglect during periods when some content has lower perceived value, or when financial resources prevent adequate attention.' (Hart 2004)

In other words, the jury is still out.

From a risk management perspective, both the likelihood and consequences of loss of digital information are significant.

Paolo Usai, the Director of Australia's Screensound, raised some important points in relation to systems for preserving digital information at the international Web Archiving Conference in November 2004. (National Library of Australia 2004)

As he called for future systems of preservation that:

  • do not require frequent updates
  • have their own metadata
  • are not reliant on electricity to make them accessible,

one cannot help but recall the microfilm and the simple magnifying glass.

At the same time microfilm has some clear limitations. Its strict linear nature can make it tedious to access and black and white microfilm does not reproduce photographs well and generally not colour.

Digitisation and access
Indisputably, digitisation is the key to enhanced access. Digitisation has increased our means to make our collections of newspapers more readily accessible through the world wide web and also by adding advanced search features such as OCR to access the text information within.

The hybrid approach
Right now, by combining the approaches of digitisation and microfilming, it is possible to get the best of both worlds. Right now, we are advocating that microfilm is the prudent medium for long term preservation, and digitisation the medium for access.

Preservation microfilming reduces the potential of significant risk of loss inherent in both the nature of the acidic hard copy papers and in the digital repositories which need to be systematically backed up, refreshed and/or migrated to ensure their ongoing accessibility.

Microfilm then becomes an integral step in the expanding future of newspaper digitisation programs.

In looking at the hybrid approach the primary method to date has been to film first and scan from the film. This approach supports the goals of the IFLA Newspapers Section Strategic Plan especially Professional Priorities:

  • (g) Preserving our intellectual heritage, and
  • (i) Promoting standards, guidelines and best practice.

It is clear from the content and debate covered at the Business meeting held immediately prior to this Conference that the heart and passion so evident in the heritage micrographics' industry over the years is alive and well in the Committee Members of the IFLA Newspapers Section.

Andy's company, New Zealand Micrographics, has been involved in some major newspaper projects such as:

Both of these very popular projects were collaborative initiatives and involved microfilm of varying vintage and quality, scanned with OCR in mind. Both were hosted with Greenstone Digital Library Software developed by New Zealand's University of Waikato - which was recently featured in the latest IFLA Newspapers Section Newsletter.

A third project New Zealand Micrographics has had heavy involvement in over the past few years is:

Ross Harvey has remonstrated the benefits of Papers Past a short while ago, whilst Dave Adams and Clark Stiles described this National Library of New Zealand initiative in good detail yesterday.

This Conference is very much about sharing knowledge. Other examples (of a variety of projects) where you might be able to compare notes with your peers when considering a digitisation project can be found at http://ndf.natlib.govt.nz .

These newspaper scanning projects in particular gave the company ample opportunity to review its microfilming practices with a view to improving their potential for scanning.

Microfilming for digitisation
Andy has noted some distinct differences in his company's approach to that recommended by the Research Libraries Group's Guidelines for microfilming for digitisation, (Meyer 2003) and to a lesser extent the IFLA Newspapers Section's Microfilming for Digitisation and Optical Character Recognition - Supplement to Guidelines (IFLA 2002).

Andy supports the caveats of 'the commonsense approach' in the IFLA document.
The following is a brief summary of where his practical experience intersects with this Supplement and he welcomes further feedback or debate on these issues.

The scope of the IFLA Supplement identifies that it reflects the 'present situation' (in fact ~2002) and both documents imply current microfilm can be 'an efficient and economically favourable intermediary in a digitisation programme if this aspect is considered during the microfilming process.' Andy contends that historical microfilm (produced purely with the intention of preserving the intellectual content of the document) can offer the same benefits.

Since Cornell University's seminal work in the mid-90's regarding its Digital to Microfilm Conversion Project (Kenney 1996), technology has advanced so that the current crop of microform scanners are quite capable of rendering sufficient detail from reasonable film, as opposed to having to go (back) to the original.

This is an area ripe for further research and development, in doing this we must endeavour to compare the scan systems on a 'level playing field.'

In an age when the best chance of funding for preservation microfilming projects is for custodians to insist it is part of a digitisation project - as a prudent risk management feature - it is sadly ironic that the bulk of the recommendations in the IFLA Supplement, if taken up, will add significant costs to the imaging budget, through slower filming rates, increased preparation time, more difficult quality control and so on.

As technology in scanning has advanced, Andy's belief is that we should put our energies into the production of the traditional quality microfilming practices - that is consistent density, excellent resolution (and the use of the 'Quality Index') and good tonal representation of pictorial content …. and let the microform scanner & software manufacturers worry about automated detection of images from microfilm. Do not spend hours collating documents into uniform blocks, leave in-film targeting in situ and do "duplicate (intentional second) exposures"

Despite some misgivings about some of the recommendations, we concur with the final paragraph of the IFLA Supplement:

'Do it once, do it right. Technology will surely remove [and is removing] some, if not all, of our present obstacles. If a quality program for microfilming is established as a part of the library strategy, it will make things easier for the future digitisation activities. Microfilming should be considered as a part of the library's digital [or 'imaging'] program.' (IFLA 2002)

This risk-averse initiative I think we should all endorse.

Both of these guidelines have been produced to identify to traditional microfilmers the 'tips and tricks' that may assist automated scanning of rolls of microfilm.

However there is good news. With recent technological advances in scanning there is now less need to make a number of the recommended modifications to microfilming practices. It is likely that these can be more easily handled by the microform scanning staff, in conjunction with microform QA staff if necessary. .

Andy has taken up the challenge by Robin Dale, Editor of the RLG Guidelines, to share his practical experiences in our part of the world - and welcomes the opportunity to discuss them with you further.

He would like to propose two suggestions for further research and development:

  1. Consider the use of notches or small holes to identify areas of potential difficulty on the film. These could be relatively easily detected as part of the scan process and files annotated accordingly for closer scrutiny.
  2. Consider the use of colour microfilm, as suggested in Goal Six of IFLA Newspapers Section Strategic plan 2004-2005. However, develop this program alongside the production of a 2nd generation Preservation Duplicate Master as an 'extended-term' greyscale polyester microfilm. Logically, with modern papers, we might consider the production of COM colour film using files direct from the publisher, if they can be formatted to suit the COM machines of the time, or scanned direct from the originals.

COM as a hybrid imaging technology
One alternative to scanning from microfilm is to digitise first (or perhaps take 'born-digital' material) and output that to microform via a Computer Output to Microform (COM) device. Today it is often a viable method of hybrid imaging, certainly with commercial documents.

Interpretation of the quality of the COM image is difficult to assess in the context of image quality using traditional microfilming methods.

Andy's experience is that he increasingly finds himself relying the modern equivalent of traditional microfilming 'Quality Index' methodology; that is to 'read the fine print'! If the output COM film can resolve a four point font then it has passed a reasonable test for resolving power. Of course, an assessment should not be made without reference to the original document and the scanned image first.

One of the main barriers to COM for newspaper imaging is the issue of resolution and tonal representation. In short COM devices struggle to render fine detail of large format material well on 35mm film.

Andy believes it is fair to say the same applies to newspapers - both old and new. We can achieve good results scanning older newspapers and outputting the digital images to COM, albeit scaling may be necessary.

However the cost of scanning and handling often-frail documents in the first instance is often too expensive. Wide-format sheet-fed scanners shine too much light through the page increasing the problem of bleed-through and necessitating the backing of the newspaper pages. Thus a (slower) overhead scanner (perhaps a hybrid scanner/microfilm camera) may have to be used, these are becoming more widespread, but are invariably more expensive to operate.

In short traditional filming methods are often cheaper, even when scanning film thereafter is included. With the recent technological advances in scanning microfilm, comparative quality can be achieved.

In the past five years, for modern 'born-digital' material, the issues are somewhat more complex. Proprietary print file formats were initially a problem; nowadays a PDF file is easy to produce by a willing printer/publisher for use on the COM device, but coordination can be difficult when a newspaper uses more than source to produce its publication - which may consist of the main body of news, advertising & classifieds, glossy or magazine inserts, leaflets etc

Another complexity is that colour photographs embedded in the files can be difficult to render on monochrome microfilm on some recorders.

On the other hand, a key benefit of COM is the ability for metadata captured at or during the scan process to be inserted in each frame if desired.

Digital to microfilm conversion is a seductive area in its infancy, rapidly attracting worldwide attention that is ripe for further research and development. A few micrographic bureaux in the region are already working in this area.

Andy summarises: 'As someone who earns his income far more readily from digitisation than microfilming I could not put my hand on my heart and risk a preservation imaging solution that only involves digitisation. Most of you in this audience are stewards of significant cultural collections, and until we reach the pivotal point when digitisation longevity issues will be resolved it behoves us to not rest on our laurels and promote the 'access' solution as a likely long-term keeping solution.'

As Alan Smith, the Director of the State Library of South Australia recently commented: 'Our greatest ambition as stewards and custodians of our respective nations' documentary heritage is to pass the collection on to the next custodian better than we received it ourselves, ….whilst providing access is essential - too much access may deny future access…' (Smith 2005)

Training and quality assurance
Training in quality assurance is also critical to ensuring that the information and knowledge is preserved for the future to the highest level.

Within the region the National Library of Australia's PADI (Preserving Access to Digital Information) site provides a valuable educational framework and signpost relating to digital preservation issues.

Within the Asia-Pacific region we are both actively involved in working with like-minded organisations to deliver training to support quality filming and digitizing at the 'hands-on' practitioner-level and also at the 'project management/quality assurance' level.

For example, along with representatives from the Australian National Library, state libraries and archives and microfilming industry, we are both members of the Australian Course Advisory Panel that will approve the re-accreditation of the new competency-based Certificate IV in Preservation Microfilming (NTIS website). This specialised course has been taken up outside Australia and ensures that microfilming practitioners have the requisite skills and knowledge of 'best practice' techniques and international standards and guidelines, including digitising from microfilm.

IFLA - PAC, the National Library of Australia and the State Library of South Australia recently collaborated to produce training materials in preservation microfilming which have been distributed throughout the Asian-Pacific region, again with the aim of skilling practitioners in quality preservation microfilming (Brown 2003).

Singapore National Archives is emerging as a regional training centre and we have recently worked with them to provide training workshops about microfilming and digitisation quality issues to practitioners and project managers. Participants from Singapore, Brunei, New Zealand, Thailand and Vietnam attended.

In the recent past SEACAP (Southeast Asian Consortium for Access and Preservation) has highlighted the need for quality assurance in preservation microfilming and systematic bibliographic tracking of surrogates. The proceedings of its February 2000 International Meeting on Microform Preservation and Conservation Practices in Southeast Asia, at Chiang Mai University are available from the SEACAP website.

Training such as this within the region reduces the risks of poor quality preservation microfilm and digital imaging - and ensures that both access and preservation needs are met for the future.

Standards and guidelines
The ongoing revision of standards and guidelines is an integral part of risk management. We have already referred to recent technological advances in scanning that are resolving some of the issues raised in the IFLA and RLG Guidelines discussed.

Within the region, a number of the delegates at this conference participate in the review and development of standards and guidelines through membership of the Australia & New Zealand Standards Committee for Micrographics and Image Management, and as we have also been doing over the past year with the National Archives of Singapore and the Singapore Code of Practice.

Standards and best practices do exist for the creation of digital master files, but the same cannot be said for the long term care of these files. 'Since such practices are still in their infancy, it is not really feasible for a large number of the cultural heritage community to accept digitization as a viable preservation method. It would be more prudent to wait until the field is more stable.' (McAlister 2004)

Quality assurance auditing
In a related quality assurance area we have also been involved in conducting a series of audits on the micrographic and digital imaging systems at the Singapore National Archives, one of the major microfilming and imaging agencies in the region. Auditing is another tool for ensuring that systems are continuing to work to requisite quality assurance levels and is linked back to their role in maintaining the identity, integrity and quality of the copy as a trusted source of the cultural record.

In turn this has parallels with the recent push for certification of 'trusted digital repositories', which was highlighted by Robin Dale from the Research Libraries Group at the Web Archiving Conference.

Conclusion
Right now we stand at a pivotal point.

A point of rapid technological advances in digitisation that are simultaneously greatly enhancing access, and yet are also beguiling.

Paolo Usai, the Director of Australia's Screensound summed up the risks at the international Web Archiving Conference in November 2004 with his comment:

'If there is too much emphasis on access instead of preservation we lose balance and our cultural assets'

Our theme is that NOW is the opportunity to balance the two.
We are at the pivotal point of rapidly expanding newspaper digitisation programs.
We are also at a point where there is the potential to virtually and seamlessly link national and regional bibliographic lists via the world wide web.

Right now an integrated risk management approach gives the opportunity to balance preservation and access.
Such an approach links back to the four key risk factors:

  1. inherent nature of the materials
  2. quality
  3. storage
  4. bibliographic tracking of originals and surrogates.

A balanced risk management approach can be achieved through:

  • hybrid technologies, using microfilm as the preservation platform and digitisation for access
  • training and the promotion of quality assurance systems such as auditing
  • ongoing development and review of standards and guidelines
  • national and regional coordination of physical and virtual storage of the originals and microfilm and digital surrogates
  • further development and virtual linking of national and regional lists of original newspapers and their microform and digital surrogates
  • further national and regional cooperation and coordination to preserve and provide access to regional newspapers.

This is what stewardship is about. As Colin Webb summarised at the International Web Archiving Conference (National Library of Australia 2004) it is doing the best we can with what we can achieve and being watchful and active.

We are all stewards of the region's newspaper heritage. And we believe that a prudent risk management approach, doing the best we can in the NOW helps achieve this balance. No one said it would be easy.
'Stewardship is easy and inexpensive to claim; it is expensive and difficult to honor, and perhaps it will prove to be all too easy to later abdicate.' (Lynch 2004)

Time's pivotal point is NOW.

Resources are limited. The risks are higher than some might appreciate. The consequences of loss are significant. Time is rapidly running out. And the balance between preservation and access is always dynamic.

REFERENCES
Adams, D et. al. 2004, 'Addressing a risk perspective when considering digitisation as a preservation reformatting method: a response from "down under" ' Microform and Imaging Review vol. 33 no. 4, Autumn pp188-189.

Arthur, Kathleen et.al. 2004, Recognizing Digitization as a Preservation Reformatting Method. Prepared for the ARL Preservation of Research Library Materials Committee.

Brown, Heather 2003, Training in Preservation Microfilming .13 volumes. National Library of Australia, Canberra. (http://www.nla.gov.au/preserve/trainmat.html)

Drake, Karl-Magnus Report on preliminary testing on COM systems [Metamorfoze conference]

Hart, Andrew 2004, 'A critique of "Recognizing Digitization as a Preservation Reformatting Method"' Microform and Imaging Review vol. 33 no. 4, Autumn pp184-187.

Hawaiian Language Legacy Programme http://nupepa.org

Kenney, Anne & Chapman, Stephen 1996, Digital Imaging for Libraries and Archives, Cornell University Library Department of Preservation and Conservation, Ithaca, NY

IFLA Newspapers section

Langendoen, Andrea 2004, 'The Metamorfoze Preservation Program' Microform and Imaging Review, vol. 33 no.3, pp110-114.

Lynch, Clifford, 2004 in RLG DigiNews 15 August 2004 v8 no.4

The Maori Language Newspapers Project (Niupepa)

McAlister, Sheila & Trinkaus-Randall 2004, ' "Wouldn't it be prudent": Digitization as a Preservation Reformatting Method' Microform and Imaging Review vol. 33 no. 4, Autumn pp181-183. http://www.nzdl.org/niupepa

Metamorfoze team 2004, ' Digitization as a Preservation Method-Comments from the Netherlands' Microform and Imaging Review vol. 33 no. 4, Autumn pp191-194. " See also Metamorfoze (http://www.kb.nl/coop/metamorfoze/expert/drake.pdf).

Meyer, Lars & Gertz, Janet, 2003 Guidelines for microfilming to support digitisation. Research Libraries group, Mountain View, California. Available online at http://www.rlg.org/preserv/microsuppl.pdf

National Library of Australia

National Library of New Zealand. Papers Past website

National Training Information Service (NTIS) website
Provides details of the Australian course Certificate IV in Preservation Microfilming.

New Zealand Micrographics

Roy, Arundhati 1997, The God of Small Things, Flamingo, Hammersmith, London.

SEACAP (Southeast Asian Consortium for Access and Preservation) 2000, Proceedings of the International Meeting on Microform Preservation and Conservation Practices in Southeast Asia: Assessing Current Needs and Evaluating Past Projects, February 21-24, 2000, Chiang Mai University, Chiang Mai. (Available from the SEACAP website

Smith, Alan 2005 Interview. State Library of South Australia, 18 February 2005.


BIOGRAPHICAL NOTE

Heather Brown
Heather Brown is currently Preservation Manager of the State Library of South Australia, and has a keen interest in reformatting strategies - particularly the field of preservation microfilming. In collaboration with the National Library of Australia and IFLA, Heather has recently developed microfilming training materials which have been distributed throughout the Asian Pacific region under the auspices of IFLA-PAC with the aim of better preserving heritage materials through high quality microfilming techniques. http://www.nla.gov.au/preserve/trainmat.html With a background in librarianship and vocational education, Heather delivers training courses in reformatting - most recently in collaboration with Andy Fenton (NZ Micrographics) for the Singapore National Archives. Contact: Brown.Heather@slsa.sa.gov.au
Andy Fenton
Mr Fenton is the Managing Director of New Zealand Micrographic Services Limited, a leading authority in New Zealand in preservation microfilming and digitisation of heritage materials. http://www.micrographics.co.nz/. Armed with extensive experience in the areas of film processing, inspection and quality control of microforms, Mr Fenton has a wealth of experience with planetary and rotary cameras as well as hybrid/microform scanners. He was invited to represent the interests of the New Zealand Micrographics Industry on the Australia/New Zealand Joint Standards Committee (MS/4) for Information and Image Management in 1996. Extending his skill-set to commercial document scanning, Mr Fenton co-founded the Desktop Imaging in 1997; and the Heritage Materials Imaging Facility, sited in the National Library of New Zealand Building, in 2003. All three organisations have a strong focus on duty of care, standards and quality. Andy is married to Audrey (Co-Director of NZMS) and they have two children - Michael (6) and Kate (3). They live in Wellington, close to the family business, in Tawa.

About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us