Newspapers are a unique and valued source of information for historical research. The National and state libraries in Australia have always recognised their importance and have worked together to collect, preserve and provide access to newspapers through the Australian Newspaper Plan. One of the most effective strategies to both preserve and make newspaper content more accessible is to digitise and maintain them for online access.
The National Library started the Australian Newspaper Digitisation Program in 2007. As of June 2012 more than seven million pages of digitised newspapers can be accessed online through Trove, the National Library’s online search service.These newspapers cover all states and territories and from the earliest published newspaper in 1803 to the mid 20th century. There are also a few newspapers in languages other than English. The digitised content is securely stored and will be managed into the future by the National Library. Libraries, historical societies and other organisations or individuals are invited to contribute to this program, to assist in achieving the goal to provide permanent access to a wide range of newspaper titles.
Through these Guidelines the Library aims to:
provide information on how libraries and other organisations can contribute digitised newspaper content to Trove;
assist libraries and other institutions to plan their projects and prepare budget bids for digitisation of newspaper titles.
Having newspapers digitised and accessed via Trove is the best way to ensure they reach a national audience. The coverage and quality of newspaper content and the innovative delivery and infrastructure supporting permanent access has already made Trove an indispensable national resource for historical research.
With 70 million articles from over 280 newspaper titles from every Australian state and territory already delivered online, Trove provides online access to an unprecedented quantity and range of Australia’s newspaper heritage.
On Trove, users can also easily find related information in a range of other formats such as books, pictures, maps and sound recordings while researching newspapers.
All of the digitised newspapers are fully text-searchable and ways to refine results are supported. Content can be searched in the context of other Australian newspaper titles.
Trove offers the facilities to enable users to enrich and enhance the data through subject tagging, text correction and annotations.
Newspapers are digitised in a standard format which supports long-term preservation and ongoing access.
The digitisation process and how to contribute
From our experience the most efficient approach to digitise a newspaper for online access is to scan the master copy of its microfilm, then perform content analysis and apply Optical Character Recognition (OCR) to the scanned content. It is a complex undertaking requiring specialised knowledge, skills and systems support. It involves consistent application of specifications, quality assurance of the digital images and text generated by Optical Character Recognition (OCR) processing, as well as providing storage and management of large volumes of data for ongoing access.
For these reasons, the National Library prefers to undertake the scanning and OCR processing on behalf of the contributing organisation, with funds transferred to the Library to cover the costs of the work. This approach has the following advantages:
- the contributors can make use of the established information technology infrastructure and specialist knowledge of the technical requirements developed at the Library;
- efficient workflows are utilised, ensuring that specifications and standards that underpin Trove are met and human intervention is minimized;
- double handling and unnecessary overhead costs are minimized;
- the contributors can benefit from volume discounts, established processes and relationships which the National Library has with scanning and OCR contractors.
The role of the contributing organisation is therefore to initiate the digitisation process by:
- nominating a title and years for digitisation. Most titles included in Trove to date are out of copyright but content still in copyright can be included where permission to do so has been negotiated with the copyright owner by the contributing organization;
- checking the availability of microfilm for the title it would like to digitise with the relevant state or territory library and seeking permission to use it;
- contacting the National Library as soon as practicable, as it is important to ensure the approach taken is compatible with the technical requirements of Trove;
- Discussing with the National Library any specific requirements and timelines for the digitisation process. For instance the Library may be able to complete the digitisation to a set timeline or manage the project in stages. The priority assigned to contributed content will depend upon the annual workload and capacity of the Australian Newspapers Digitisation Program (ANDP) team, the size and complexity of the project.
The National Library welcomes opportunities to support funding applications by contributing organisations.
In the event that an organisation wishes to digitise newspaper content through its own project for ingest into Trove, it must discuss the project with the National Library as early as possible to ensure the digitised content meets the Library’s technical requirements for ingest.
- the content must be independently quality assessed to ensure it meets the Library’s standards and specifications;
- the required files (TIFF, XML METS/MODS, XML ALTO) must be provided on media suitable for the Library’s automated ingest system. A fee will be applied to cover data transfer, ingest costs and any National Library staff time involved in managing these processes;
- the content must have no restrictions imposed on access;
- thorough testing of content will be required, and as such, scheduling and cost issues would need to be investigated and discussed on a case-by-case basis.
Please note that for organisations wishing to take this approach the National Library can provide only limited advice and minimal assistance.
Based on the preferred approach in which organisations provide funding to the Library to manage the digitisation process and delivery of content in Trove, the charge applied by the Library is $2.00 per page plus GST. The charge will be reviewed in July each year.
This indicative cost is based on scanning an English language newspaper from microfilm, and covers the direct costs of the digitisation process – scanning, OCR processing, content analysis and quality assurance – and the full cost of staff time spent on project management. A breakdown of the costs in these categories is listed further below. The National Library will provide free-of-charge all systems infrastructure required to ingest, store and manage the digitised newspapers for long-term access.
Additional charges may apply if:
- scanning from hard copy newspapers is required;
- the language content of the newspapers is not English; or
- the contributing organisation requires its own copy of the digitised content.
Roles and responsibilities
The National Library will undertake digitisation on behalf of organisations for a charge as outlined above on the understanding that:
- unrestricted access can be provided to the content through Trove;where the source of the content (for instance, the original newspaper title) is still in copyright, the organisation contributing it must negotiate and document reproduction permission with copyright owners;
- the contributor will establish a standard Wikipedia (http://en.wikipedia.org/) page providing information on the title being processed;
- if they are a Libraries Australia member, the contributor will catalogue the title to be digitised on Libraries Australia;
- once ingested into Trove, no content will be removed or suppressed except for legal reasons;
- the National Library will acknowledge the funding source of contributed content on Trove;
- the National Library has the right to manage the content for long-term access in whatever way it considers appropriate;
- if the contributor requires a copy of the master file of contributed content, this can only be provided at the time the content is processed into Trove.
Letter of Agreement
Before the National Library can carry out a digitisation project on behalf of another organisation, it is required that a Letter of Agreement is signed by both parties. The Letter will cover the roles and responsibilities outlined above that will apply
For further information and to discuss opportunities for inclusion of content in Trove, please contact either:
The Contributor Liaison Officer
Australian Newspaper Plan
National Library of Australia
(02) 6262 1005
The relevant ANDP state and territory libraries' respresentative
Breakdown of the costs incurred by the National Library in managing a digitisation project on behalf of another library.
(i) Project management
Project set-up tasks
- Provide a cost estimate for the project to the fundingorganisation.
- Ensure the funding organisation understands the rols and respoinsibilties as outlined above.
- Confirm the availablity of the source material.
- Ensure an ISSN has been alocated to the title.
- Schedule the title for scanning and OCR processing and establish rders with the scanning and OCR contractors which set out all of the requirements and timelines for the title being processed.
- Establish budget processes.
Project oversight tasks
- Arrange inventory for source microfilm and its secure transport to scanning contractor and its return to owner.
- Arrange transport of digital images to the National Library, writing of data tapes and courier of tapes to OCR contractor and ftp transfer of OCR deliverables to the National Library.
- Undertake a pilot process with the OCR contractor to ensure correct application of specifications to the particular newspaper title.
- Establish a weekly delivery schedule and monthly reporting schedule with contractors.
- Monitor workflow and adherence to timelines and schedules with contractors.
- Monitor workflows associated with data ingest.
- Undertake trouble-shooting and clarification of requirements and specifications with contractors as required.
- Communicate with the funding organisation.
- Arrange for acknowledgement of support on Trove for the funding organisation on completion of the project
(ii) Digitisation (scanning, OCR processing, content analysis, and quality assessment)
- Per page cost of scanning, OCR processing and content analysis using the National Library’s service providers.
- Per page cost of quality assurance carried out by National Library staff.
- Cost of data tapes for transfer of digital images.
- Courier and packaging costs for transfer of microfilm and data tapes.
Items with cost absorbed by the National Library
- Provision of systems infrastructure required to ingest and manage the contributed data.
- Provision of systems infrastructure required to store and preserve the contributed data for long-term access.
- IT support for the whole cycle of digital files constituting the content of digitsed newspapers on Trove.