The first several months of the Project has involved a tremendous amount of fundamental technical, conceptual, planning and organising work.
The bulk of this work involved laborious checking and selection of "best copy" for use in the project, the development of the technical standards for production microfilming and scanning, and the development of a document control architecture for accessing and browsing the digital files. This phase culminated in the establishment of a testbed website to enable us to do some initial investigation into variations of image quality, data file size, transmission times, and use of viewer software. The testbed provided more questions than answers, and caused us to think carefully about the best ways to proceed in both establishing technical benchmarks and providing the material to scholars in a useful and accessible way.
1 BEST COPY
During the first months of the project substantial work was done, by staff
at the NLA, SLNSW and USL, on the checking of material within the project
parameters and finalising criteria for selection of "best copy" for use in
the project. This process has been much more time consuming than
anticipated, and involved quite detailed and careful checking of the titles
identified from the period 1840-45.
Matters that needed to be resolved included: criteria for "best-copy", for both print and existing microfilm; detailed identification of holdings and gaps in holdings at each site consideration for selecting titles with contemporary or significant annotations; identification and consideration of non-Ferguson titles; establishment of criteria for use of existing microfilm sets.
Consolidated tables have now been prepared for the final selection of titles for the inclusion in the project.
2 TECHNICAL SPECIFICATIONS
The preparation of the production specifications for microfilming, digital
scanning, and the file structure for the delivery of images needed to be
detailed and complete before tenders called and contracts let. This
required a full understanding of all technical aspects and a vision of how
the final images will appear as well as an understanding of how they will be
used. The Technical Advisory Group have had to work through the whole
process conceptually and technically to arrive at these specifications - and
this has taken considerable time.
The critical technical issue in the development of the specifications is the data file size of the images - the larger the file size the slower it is for the image to be retrieved across the internet, the slower to download, and the more space it will take on disks to which the images have been downloaded.
File size issues are addressed in terms of image quality, and data compression.
The specification for a broad database architecture and document control structure, which will enable ease of browsing and information retrieval, has been developed at the NLA (assisted with LC and Cornell documentation) . This control structure will specify the kind of file structure in which images must be delivered by the contractors for ease of loading into the database. This kind of structure must be able to handle and describe various kinds of pagination (including covers, flyleaves, titlepages, tables of contents, missing pages, inserts etc) as well as include information on technical specifications used and other relevant matters such as source.
3 TESTBED SITE
The need for testing the specifications led to the establishment of an
internal testbed website to check the image quality scanned at differing
image resolutions and depth. The purpose of this process was to establish
the optimum scanning specifications and data file sizes for the retrieval of
useful and clear images of the material across the internet.
It became clear through this testing process that, firstly, we needed to do much more investigation before finalising any specifications for tender, and, that, secondly the handling of large format newspapers would be a problem - the data file sizes were so large per page that the time to retrieve an image across the net was much too long, and the print version from the image was far too small.
The experience of the testbed caused the Steering Committee and the Technical Advisory Group to re-evaluate the next stages of the project.
4 NEXT STEPS
After much discussion and several joint meetings between the Steering
Committee and the Technical Advisory Group, there was agreement that:
a) More investigation was required on image quality, file sizes and network delivery before we could proceed to call for quotes for the whole project, and that we should introduce a longer testing phase into the project, working in conjunction with a company with some experience in the area.
b) We should defer the digital conversion of large format newspapers till later in the project and investigate other means of delivery (eg CD-ROM).
Testing Phase
A call has been made for expressions of interest from companies with experience in the area to work with us to investigate, in a production environment, issues of optimum image quality and network delivery. To go to full contract without fully understanding, and being confident in, the technical requirements of the project would be foolish, a waste of money, and ultimately a blow to the credibility of this initiative.
This testing phase is scheduled to begin in September and conclude in late November. Five titles have been selected - three to be filmed/scanned from the print version - the True Catholic: or Tasmanian evangelical miscellany (Jan 1843 - Jun 1843); the Colonial Observer (7 Oct 1841 - 26 Dec 1844) and the Colonial Literary Journal ( Jun 1844 - Mar 1845) - and two from existing microfilm - the Colonial Magazine and Commercial Journal (Jan 1840 - Dec 1843) and the Star (2 Mar 1844 - 23 May 1845). These titles have been selected to represent a number of technical issues, including a variety of page sizes.
Copies of the Expressions of Interest documentation are available.
Newspapers
The newspapers of the period represent a unique record of the settlements, but their physical size presents particular problems in digitisation, especially in the delivery of the images and print reproduction. In investigating these problems we consulted with several digital projects working with newspapers, and all (at least for now) rejected using the net as the delivery means of images of large format newspapers for the same problems we have encountered.
The document structure developed for the project can be used for organising and browsing newspaper pages, and this can be used for the provision of images on CD- ROM. This seems the likely option for our situation, but further investigation is required.
5 FICTION
The microfilming and scanning of the four works of fiction were completed
at beginning of year and sent to Monash so that they could be worked on
separately while the technical issues surrounding the journals and
newspapers could be addressed. The appropriate software and hardware to
assist with the conversion of the scanned pages via optical character
recognition (ocr) to ASCII has purchased and work has begun.
The document structure of the fiction data files will be different from that of the journals because of the different structure of the novel from journals and the plans to be able to mount and link the ASCII form of the page and the image for browsing.
6 STAFFING AND SUPPORT
The bulk of the work has been carried out at the SLNSW and the NLA , and
funds have been made available to both institutions to assist with the
co-ordination of activities from both collections. It must be acknowledged,
however, that each institution has contributed a great deal of management
time to date on the project through the members of the Steering Committee
and Technical Advisory Group.
Specific acknowledgment is due, at the SLNSW to Alan Howell (convenor of the Technical Advisory Group), Rebecca Thomas and Andrew Heath, at the NLA to Diana Dack, Juta Hosel and Laurie Salter, and at Sydney Uni Library, Julie Price.
Acknowledgment also needs to be made of the assistance and advice from Cornell University Library, and documentation from Cornell and the Library of Congress digital library projects. Their work has helped us in understanding and addressing several technical specification issues.
7 BUDGET
These are details of expenditure, and anticipated commitments, to date
against the ARC funds. Other expenditures/contributions from the core
partners have not been included, but will form part the final financial report.
Our approach to the budget has been extremely tight, to keep adequate funds aside for the large commitments that will need to be made.
Of these commitments, we know that the digital production costs will be expensive (more than we had budgeted in the ARC submission), due in part to the high quality control standards we are setting for the production work, and the requirement that the images be provided in a specified file structure. There will also be some considerable costs in setting up the database architecture and loading, testing and linking the indexed digital files as they are received at the NLA. The test phase itself will also provide additional costs. Some of these commitments have been estimated, while others will depend on quotations received.
| Funds (ARC) | 300,000 | |
| Expenditure to date: | ||
| Equipment | 6,510 | |
| Software(OCR) | 940 | |
| Coordination support | 10,757 | |
| Filming/scanning (fiction) | 1,711 | |
| Travel | 719 | |
| 20,637 | ||
| Commitments (Estimates): | ||
| Equipment (NLA server) | 32,000 | |
| Coordination support | 15,000 | |
| Development/maintenance of database | ||
| architecture and image control, | ||
| retrieval and management (NLA) | 75,000 | |
| ASCII conversion support (Monash) | 5,300 | |
| Test phase (quotes to be received) | ||
| 127,300 | ||
| Total | 147,937 | |
29 August, 1996