Rare Maps Digitisation Project
Project Report
This report documents a pilot project to digitise 25 maps from the Librarys Rex Nan Kivell rare map collection. It builds on the work of a trial project on rare maps digitisation carried out in 1998.
Impetus for the Library to investigate the feasibility of scanning its rare map collection came from the appearance of a number of projects on the Internet by overseas map libraries, notably the Library of Congress, during the early 1990s.
Most of the projects appeared to be scans of photographic images of the original material with the occasional use of direct scan images. A review of the literature raised a number of issues, the principal concerns relating to indirect verses direct scanning, and resolution levels. Apart from a preservation scan project on the Utzon Opera House plans by the Mitchell Library no similar undertaking had been considered by any other Australian library for their early map collections.
A talk and demonstration of the Library of Congress project by the Head of the Geography and Map Division, Ralph Ehrenberg, at the National Library in 1997 encouraged the establishment of a trial project which began in March 1998.
1998 Trial Project
The aim of the trial project was fivefold, with particular emphasis on the first three.
- To investigate the feasibility of digitising parts of the
National Library of Australias map collections in order to make the
collections more accessible to all Australians and overseas scholars through
the Internet.
- To provide an alternative copy of rare and valuable
material.
- To explore the issues relating to more routine digitisation of
the map collection.
- To provide leadership to the map community as the project would
be the first in Australia to provide illustrated and catalogue records of rare
maps held in an Australian collection
- To explore the extent of co-operation possible with map publishers in Australia on the project, particularly in relation to technical issues and publications.
For the trial project, 25 maps were selected from the Rex Nan Kivell rare map collection. These were used to investigate:
- The most effective means of creating high quality digital scans
of rare maps;
- Appropriate resolution levels required to provide satisfactory
viewing levels; and
- Linking digital scans to bibliographic records and mounting a test database on the Internet.
The trial established that direct scanning from the original using a high-quality digital camera at minimum 300dpi level appeared to give the best result. Links to bibliographic records and presentation of maps on the Internet could not be completed during the trial due to a number of software problems. Further work was also needed to establish procedures for a full rare maps digitisation program.
Pilot Project
A pilot project was then proposed which would examine in more detail the technical issues relating to direct digitisation of the maps, linking to their bibliographic records and mounting on the Librarys Information Server. The maps would be digitised using a high quality digital camera and procedures developed for routine digitisation of the maps. The pilot project commenced in August 1999.
As an adjunct to the project five map series indexes relating to the Australian Capital Territory (ACT) were also selected, with linkages provided to the bibliographic records of the relevant series, for mounting on the Librarys Information Server.
Methodology
The following sections document the methodology used to capture the rare maps and map series indexes in electronic form during the pilot project.
Map Selection
The Rex Nan Kivell rare map collection was chosen as a suitable map collection for scanning and mounting on the Librarys server. This collection was nominated for its small size (around 1,000 items), the generally good condition of the maps and the range of maps in terms of size, age range and colour type. Some other parts of the Rex Nan Kivell collection in the Library have already been scanned, notably in the Images 1 project, and the scanning of the map component would complement the digital record of this collectors activities. The collection has also been completely catalogued on to Kinetica.
The same 25 items were used for both the trial and pilot projects. These items represent the range of material in not only the Rex Nan Kivell collection but also the other three rare map collections in the Library. Items giving a good representation of size, black and white/colour, condition and particular problems (eg dissected and mounted, bound, manuscription annotations) were chosen. The rare maps selected for digitisation are listed on the web site at: http://www.nla.gov.au/rmaps/rmapslist.html.
The five map series indexes selected were chosen based on the complexity, legibility and size of the indexes and are listed on the web site at: http://www.nla.gov.au/mapseries/.
Handling of material
Digitisation presents preservation risks to all material. For valuable rare maps the risks require special attention, because of the characteristics of the material and because the level of damage that would be acceptable is virtually zero.
Potential risks
The following risks were identified for the material being digitised:
- Tests demonstrated that digital cameras would produce more
detailed images than flatbed scanners for this material. Although digital
cameras require special rigs to support the source object or camera or both,
presenting some potential risks to the material (see below), these arrangements
are usually preferable to manipulating fragile or large items onto flatbed
scanners, especially if their size makes it necessary to take a number of
captures, segment by segment. Both are definitely preferable to drum or
pull-through scanners which generally present unacceptable risks of damage
(although they have been used in well-controlled projects such as the State
Library of NSWs digital capture of the Utzon plans of the Sydney Opera
House).
- The size of maps often makes them hard to handle safely,
demanding care in handling at all stages including selection, assessment,
extraction from storage, transport to the capture station, temporary storage
before and after capture, mounting for capture, demounting after capture, and
return to storage. Size also often makes maps hard to store flat, so larger
maps are often found stored rolled or folded, requiring special treatment
before they can be presented flat for capture, and if they need to be returned
to the same storage arrangements.
- Digitisation processes require the map to be held flat and
parallel to a scanning surface. To achieve this where the map is larger than
can be accommodated under the camera usually requires the map to be held flat
against a vertical surface, using magnetic strips, vacuum frame, removable
self-adhesive strips, or something similar. All of these pose risks of damage
needing to be recognised, assessed and managed.
- Finally, all light-based copying processes mean exposing the map to light, and potentially to heat from the light source. Achieving high resolution digital images in acceptable timeframes requires high light levels and relatively long exposure times (some minutes per item).
Risk mitigation
The following procedures were followed during the pilot project to minimise risks to material:
- All selected materials were assessed by Maps and Conservation
staff to plan the way they should be handled, treated, transported, and stored.
Instructions on their handling were prepared and their use was supervised at
all times to minimise the dangers of damage in handling.
- Conducting all capture processes on the Librarys
premises obviated the need for special transport containers and reduced the
number of critical handling points.
- Most of the selected maps were in good physical condition and
did not require more than careful handling.
- Maps were transported in Mylar folders on trolleys.
- During capture, maps were held by vacuum.
- To minimise exposure to heat and UV associated with long exposure times, the contractor used a lighting system consisting of four halogen lights of 1500W each with power conditions to stabilise light intensities during image capture.
Workflows
Workflows for the production project should include:
- assessment of condition;
- transport on trolleys that provide full support;
- conservation preparation (including relaxing folded, rolled or
creased items before flattening);
- supervision of capture by NLA staff;
- improved arrangements for temporarily storing maps before and
after capture; and
- brief assessment of condition following capture.
Ideally, the maps should also be sorted into batches of like sizes to minimise the camera adjustment time. However this would require significant additional handling as well as sorting and storage space, and would only work with a recordkeeping system that allowed items to be returned to their original order. These requirements are considered to be unachievable at this stage.
Digitisation Equipment
This section documents the equipment and processes used to digitise the material.
Image Capture
The 25 rare maps were captured over a period of three and a half days by a contractor working at the Library with assistance from NLA staff. Almost one day was spent in setting up the equipment and the time to prepare maps, capture and write each image to disk was approximately 25 minutes. It is believed that this time could be reduced to about 15 minutes per image if a dedicated digitisation laboratory was available at the Library.
Image capture was completed using a Phase One digital scanning back with a scanning resolution of 6,000 x 8,400 pixels and 36 bit internal color depth.
The contractor supplied the Phase One digital scanning back. The Phase One camera captures 6,000 x 8,400 pixels which corresponds to items of 50.8 cm x 71.1 cm (20 x 28) at 300 dpi resolution. Maps of this size or less could be captured via one digital image. For maps larger than this size, of which there were seven, multiple captures were required to digitise the entire image. For future digitisation work with large maps, it is recommended that the latest digital camera is used with the highest scanning resolution available so that large maps can be captured via one scan. Since the digitisation work in this project was completed, a new model of the Phase One digital camera has been released which is able to scan items up to 88.9 cm x 106.7 cm (35 x 42) in a single scan at 300 dpi resolution. This is the Phase One PowerPhase FX scanning back (with a scanning resolution of 10,500 x 12,600 pixels). If this camera had been used for the project then all but four of the rare maps could have been captured via a single digital scan.
The Phase One scanning back performed very well. It was accurate and relatively fast with exposure times of about 10 minutes per image. Software allowed precise control of colour and resolution. The scanning back was colour calibrated at the beginning of every photographic session. An International Color Consortium (ICC) colour profile, created for the back by the contractor, allowed for precise colour translation between the scanning back and the computer.
The Phase One scanning back was combined with a Sinar camera provided by Photographic Services. It is a very high quality camera and together with Phase One scanning back made an excellent choice of equipment.
Special, high-resolution lenses are required for digital capture to deliver the best quality. The contractor supplied his own lens, the Macro Sinaron SE 360mm lens, for digital capture which were optimised for digital photography. Photographic Services also have a Macro Sinaron SE 180mm lens that can be used for digital capture.
The contractor built a special trolley to accommodate the camera and digital back. The trolley allowed for very precise camera movements and helped to speed up the process of capturing images.
Four halogen lights of 1500W each were needed to deliver enough light required for the capture. With long exposure times some maps may require professional assessment before being exposed to such light levels. Lights were supplied by the contractor.
To maintain very stable light levels during the image capture power conditioners were used. They were hired from a third party for the duration of the project.
A G3 Macintosh computer with an LaCie 19 monitor was used for controlling the scanning back and temporary storage of images. The computer and monitor were colour calibrated to Commission internationale de l'éclairage (CIE) D65 standard and used ICC colour profiles to convert images from the Phase One to Apple colour space.
The digitisation of the map series indexes involved scanning the original index sheets on a flat-bed A4 scanner at 300dpi resolution. Larger index sheets were scanned in sections on the A4 scanner but the time involved in stitching the images back together in the image editing software rules this out as a realistic work practice. Ideally, map indexes should be digitised using a digital camera so that they can be captured via a single scan.
Some work was done with digitising the existing hand annotated map indexes. However, the poor quality of the photocopy reproduction of the base index maps, in many cases, reduces their legibility and usefulness in a web-based context.
Image processing
Scanning of large maps at relatively high resolution (300 dpi) produces large image files. The master TIFF digital image files for the largest maps that were captured are over 600 MB in size. Processing such files using Adobe Photoshop requires a relatively powerful computer.
A twin CPU 500 MHz computer with 1 GB RAM was purchased for the project to undertake image processing. It operated according to expectations and greatly increased the efficiency of processing TIFF images. Experience indicated that TIFF files up to 300MB could be opened, processed and saved very quickly using the system.
However, when using Adobe Photoshop for stitching the larger maps, such as NK646, the working image file was over 1GB in size and resulted in greatly increased processing times. For example, the processing time taken for relatively simple operations on the larger maps, such as file saves, was in the region of 15 to 20 minutes.
While Photoshop proved to be an effective application for processing the images, its need for large amounts of spare scratch disk space, especially when processing the larger image files, meant that the application regularly ran out of virtual memory and failed. To prevent this from happening again, at least 50GB of internal hard disk storage capacity should be purchased and access to networked file storage should be arranged.
Several CDs were created for temporary storage of image files and to free up space on the computers hard disk. This proved to be an inefficient and time-consuming practice. Access to fast networked file storage would enhance the efficiency of the overall workflow process.
Digitisation Software
This section documents the software used in the digitisation process.
Image Capture
The Phase One scanning backs accompanying image capture software controlled all aspects of digital capture except framing and focusing. The software is well designed and implements extensive colour control modules.
The Uniform Light Tool was used to overcome problems created by uneven lighting of maps. It is virtually impossible to physically achieve even lighting across a large area, so the Uniform Light Tool proved to be extremely useful especially for images that needed to be stitched.
Adobe Photoshop v 5.02 was used to check scanned images and to convert some of the images to the Adobe RGB colour space.
Image processing
Adobe Photoshop was selected for the image processing phase of the project. Not withstanding the very large image files generated during the image stitching process, the application proved to be very effective and justifies its status as the imaging industrys defacto standard for image processing.
Before selecting Photoshop for the image stitching process, a number of applications were tested to see if more automated processes of referencing and stitching images could be achieved. The inability to create accurate registration marks prevented GIS applications like ER Mapper from being used to automate the stitching process.
Photoshops layer and transparency functions enabled image tiles to be aligned with a high degree of accuracy. However, stitching the image tiles required a great deal of time and operator patience. For instance, it took over five hours to stitch the six separate image tiles of the Hunter River map (NK646) into its final form. By way of comparison, single TIFF map images took only 10 to 20 minutes to process. Because of the excessive time required for image stitching, the Indian Ocean map (NK9522) which was captured as 9 tiles was not stitched. It is recommended that image stitching is not undertaken as a routine part of the full map digitisation workflow as the process is too time intensive. As the scanning resolution of digital cameras increases, large maps will be able to be captured in a single scanned image.
The map series index images were also processed using Adobe Photoshop. They were first cropped and adjusted for optimal brightness and contrast. The transparent color overlays, indicating the maps held in the Library collection, were then created using the functionality available in Photoshop. Essentially, the collection index information is maintained in its own layer where it can be updated and corrected as required separate from base index map. The maps series index master image files were generated by saving a 24 bit RGB TIFF version 6.0 file from Photoshop.
Digitisation - Standards
Image Capture
Computers and the digital scanning back used in the image capture were colour calibrated to CIE D65 standards and were using ICC colour profiles.
Images were captured at high spatial resolution (300 dpi at 100%), 36-bit RGB colour resolution and stored as 24-bit RGB TIFF version 6.0 files.
Image processing
The 300 dpi minimum resolution set for the maps appears to be justified on the basis of a visual inspection of the final images. However, it is believed that a case can be argued for a higher resolution of 600 dpi in selected cases where the original image is small, finely detailed or unclear. In such cases, curatorial directions should be provided to the digitisation operator to resolve the optimum image capture resolution.
Digital master images were transferred to a network accessible RAID storage device and also written to CDs. Digital master files should be stored in uncompressed form as 24-bit RGB TIFF version 6.0 files for preservation reasons and to ensure ongoing access.
Image Compression
The project team examined several technologies for delivering large images over the web. These were:
- MrSID Image Server from Lizard Technologies (http://www.lizardtech.com)
- Image Web Server from ERMapper (http://www.earthetc.com)
- E-image from HMR Inc. (http://www.hmrinc.com/en/default.htm)
The E-image option was not evaluated in detail as it was discovered towards the end of the project. It supports on-the-fly compression for delivery of TIFF images over the Internet and requires an NT based web server.
A trial of compression using MrSID and the Image Web Server was undertaken. Compression ratios of about 20 times were achieved with both systems without appreciable loss of quality. Both compression techniques use wavelet compression and both were considered to be appropriate for delivery of large map images over the web. An advantage of the MrSID solution is that users can access images without a browser plug-in and the server operates on either a UNIX or NT platform. The Image Web Server from ERMapper requires a browser plug-in and the server is only currently available for the NT platform. The Library purchased a licence of the MrSID compression software for the trial project and it was decided to also use MrSID for the pilot project. An upgrade of the MrSID compression software to the latest version was purchased at a cost of US $450. All of the images created in the pilot project, including the 25 rare maps and the 5 map series indexes, were compressed at 22:1 compression ratio using the MrSID compression software for delivery over the web.
A watching brief should be maintained for alternate wavelet compression systems and standards based file formats such as lossless JPEG 2000 should be explored as alternatives to proprietary formats.
Web Site
A home page and supporting documentation are included on the Rare Maps Digitisation Project web site at: http://www.nla.gov.au/rmaps/. Web pages for each map were created using metadata from MARC catalogue records for each map obtained from Kinetica. JPEG thumbnails were generated from the MrSID files of each map automatically by the MrSID Image Server software. Links were created from the JPEG thumbnails to the MrSID version of the map images.
The MrSID Image Server allows users to zoom and pan the map images either with or without using browser plug-in software. The plug-in software provides additional functionality such as dynamic panning through an image grab and drag process and zoom to a user defined region. The site provides a link to the LizardTech web site for users who wish to download the plug-in. LizardTech has also recently released a Java applet viewer with similar functionality to the plug-in.
Prints of Digital Images
The quality of digital images captured was tested by producing colour prints at the same size as the originals for two of the larger digitised maps. The maps were NK646, the Hunter River Map, and NK5928, a map of NSW and Queensland. A0 colour prints were produced by a commercial printer from the TIFF master images supplied on CD.
The prints were produced at 200 dpi but the quality of the output was acceptable even at this lower resolution. Comparison with the original maps highlighted some colour matching problems which could be fixed by some simple colour calibration test prints in cooperation with the supplier.
Results
The image quality of the digitised maps is outstanding. With the Photoshop unsharp mask filter applied to the images the detail and fidelity of the images is excellent. After MrSID compression (22:1), the images are virtually un-degraded and would be very useful for visual researchers, curators and so on. Access to such high quality digital images is of use to researchers and other library users and greatly enhances the profile of the NLA in making its visual collections available to the public. Using the MrSID compression software, access to large, high resolution images is possible over the web with adequate performance even over a low speed 28.8 kbs modem.
The results of the image stitching process are mixed. The large maps, such as NK646, produce large 600MB TIFF files. These maps were stitched along backing tape joints so slight variations between each image tile were obscured. However, in the case of the Brisbane Bridge plan (NK 2456/151), slight variations between the images prevented an exact alignment of the image tiles. From preservation and curatorial perspectives, a 1-pixel misalignment, while not obvious to most users, is sufficient to invalidate the stitching methodology. Preservation issues aside, the excessive time taken to stitch the image tiles together rules it out as a realistic production option.
The digital capture of the images and the digital post-processing phases should be functionally separated for maximum through-put. Images should be digitally captured in one pass, using the highest resolution device available. The highest capacity Phase One system currently produces 300MB plus files suitable to capture over 90% of the NLA rare map collections at 300dpi. The higher costs of this system can be justified by abandoning the high cost image stitching process. For maps over 88.9 cm x 106.7 cm (35 x 42) in size, current digital cameras do not provide an effective method of capture at high resolution due to the need for image stitching. For such maps entire images could be captured at a lower resolution or their capture should deferred until new models of digital cameras are released with higher scanning resolutions.
Images obtained by digitising the maps are of a very high quality. Spatial resolutions achieved, especially with multi-capture of large maps, rival or exceed that possible with traditional techniques. Due to much higher than film dynamic range of the scanning back, images proved to be clearer and contain more detail than those on film do.
The Uniform Light Tool software allows even lighting across the entire image something extremely difficult to achieve in traditional photography.
Use of ICC colour profiles provides us with greater colour fidelity than is possible to accomplish with film. However, ICC profiling is still not uniformly adopted that may create colour deviations when images are being printed or used in other ways by a third party. It is expected that this problem will be solved to a great extent with the arrival of new operating systems like Windows 2000.
The digital image capture is a fast process. Most problems arose from the need for transfer of images from the computers hard disk to other storage media. It proved to be a tedious and slow process using JAZ and CD discs. Also, risk of losing or corrupting images increases using this process. Access to a high capacity networked disk storage device with RAID disk redundancy and backup and a fast network (at least 100 Mbps) is recommended to help solve data storage problems.
Use of a specialist contractor for image capture was a mostly successful experience. Contractors of sufficient skill and experience to use high resolution digital cameras are uncommon and expensive. The contractor used for this project charged $A2,000 per day plus equipment charges. The time taken for the project expanded from a quotation of 2 days and a cost of $A4,850 to an actual time taken of 3.5 days and $A7,450. The additional costs were incurred largely due to the lack of a permanent digitisation facility at the Library and the time taken to set up and prepare for the work to commence (about 1 day). For the production project it is expected that these costs would be significantly lower per item due to economies of scale, streamlined workflow processes and use of dedicated facilities for digitisation. Quality and management control are issues to be addressed if digitisation work is outsourced.
Use of MrSID for image compression was successful with master TIFF images being compressed at 22:1 without loss of image quality. Portions of images at various resolutions can be dynamically and quickly delivered over the web using the MrSID server. The latest version of the MrSID server software also has the capacity to overlay an image such as a copyright message or logo on all images delivered over the web. This feature was investigated but not used in the pilot project but would be available if the Library needed it in the future. Zooming was a successful feature of the MrSID server to reveal fine details of map images. However, this feature caused some problems with the map series indexes as index numbers along the edge of the image were hidden as a user zoomed in. This issue will need to be resolved before regular scanning of the Library's map series indexes commences.
With regard to the map series indexes, it is estimated that an operator could master the techniques described in this report to create the index images after a day or half-day of training (depending upon their prior computer experience). The computer operator would need access to a computer workstation capable of handling large image files efficiently. The major benefits of the image processing approach adopted for the map series indexes are:
- Base index maps are digitally preserved and can be reproduced at a high quality in case of their loss or damage;
- New editions of the map indexes can be generated as information about the relevant map series is added or corrected; and
- New web-based services can be built upon the digitised map indexes.
From Pilot to Production
The project team recommends that the Library adopt a collection-based approach to rare maps digitisation. The Rex Nan Kivell collection should be the initial collection to be digitised in the 2000-2001 financial year. A working title for this activity is Project Terra Australis. Workflow processes need to be developed and documented to avoid bottlenecks in the digitisation process. Assuming that each master image from this collection averages 300 MB and that 20 images can be captured per day then about 30 GB of data will be generated per week of digitisation. The time taken to capture 1,000 items from the Rex Nan Kivell collection would be approximately 10 weeks generating 300 GB of master image files and approximately 15 GB of compressed and lower resolution image files. A substantially longer period of time would be required to undertake the necessary image processing, web site development, catalogue updating etc subsequent to image capture.
The digitisation of the rare map collection should be undertaken at the Library for preservation reasons so that handling of original items can be monitored and risks inherent in transport of material off-site are avoided. A dedicated digitisation laboratory is required to be set up at the Library with all the necessary equipment for digitisation to occur.
Conclusion
The rare maps digitisation project has proven the feasibility and worth of the use of digital cameras for image capture and compression software for delivery of large and high quality map images over the Internet. It has also highlighted the need for technical facilities to support digitisation as a routine process at the Library by contractors or Library staff.
Recommendations
- That the Library fund the digitisation of the Rex Nan Kivell
rare map collection in the 2000/2001 financial year conforming to the technical
procedures defined in this report.
- That digitisation of map material from the Librarys
collection be undertaken using a high-quality digital camera at a minimum of
300 dpi. That resolutions of 600 dpi be considered for highly detailed material
on a case by case basis.
- That tiling and stitching of digital images of large format
maps be avoided in digital imaging workflows through selection of a camera of
sufficient resolution to capture entire maps in a single image.
- That master digital images of maps be stored for long-term
preservation in uncompressed form on a network accessible RAID storage device
and on CD in 24 bit RGB TIFF version 6.0 format.
- That MrSID software be used to compress map images for delivery
over the web using a compression ratio of 22:1.
- That the Library develop an appropriate solution to the
problem of loss of sheet identification detail (along the margins of the index)
when zooming in on large format map series indexes.
- That to ensure correct handling and preservation of original materials, digitisation is undertaken in-house at the Library under supervision of Library collection managers.
Further developments
Since September 2001 the Library has commenced a regular program of scanning its rare map collections. Images are available through a highlighted url address in the catalogue records for each item. Further information on this project may be obtained at the following website : http://www.nla.gov.au/digicoll.
