Paper written by Colin Webb, Manager, Information Preservation.
The National Library of Australia has embarked on a program of digital conversion of its large and important Oral History collections. The Library has been using and investigating digital recording and editing facilities since 1991. Responding to the benefits offered by digital technology, and recognising the need to transfer from analogue to digital to maintain access, the Library initiated the purchase of a digital system in early 1996. The new system addresses the medium term requirements of digital audio archiving; its design also includes the possibility of automated migration to later digital archiving systems, including possible mass storage.
The system, managed by the Sound Preservation and Technical Services (SPATS) unit - part of the Library's Preservation Services function - uses a network of computer hard disk-based Digital Audio Workstations writing to recordable CD (CD-R), analogue reel-to-reel tape and analogue access cassettes. As procedures develop, it is likely that the analogue components of the current system will be dropped in favour of full digitisation. The size of the collection and the conversion task mean that even with the full digitisation of current procedures the collection is likely to contain analogue tapes for many years to come.
This paper provides some of the context for this project and the planning processes involved in the decision to digitise. The process is then described, with some discussion of advantages and disadvantages, and some issues for the future.
Sound preservation activities in the Library are primarily undertaken against corporate objectives concerning the collections and services to the community:
In this context, SPATS provides a comprehensive internal audio preservation service dealing mainly with the technologies and formats found in the Library's Oral History collections. The collections, currently containing about 30,000 hours of original recordings, cover a very wide range of notable Australians, social history and folklore projects from the late 1940s onwards. It is arguably the most important oral history collection in the country.
The great majority of items in the collection are unique, ie the Library holds the only copy. The collections were previously almost wholly carried on reel-to-reel and cassette tapes with cellulose acetate, PVC and polyester bases, with a variety of binders. There are examples of materials right across the 50 year history of magnetic tape recording. Many of the tapes are old, unstable and deteriorating; all require access and safety copies. To deal with this SPATS preservation-copies around 1,500 hours of recorded material per year. Until recently this was onto a stable analogue reel-to-reel format.
Facilities and skills have developed over a number of years to address the Library's objectives, by servicing these collections and serving its clients. Major capabilities in the preservation of audio recordings include:
From the mid-1980s the Library developed and adopted an internal set of technical standards for production of analogue originals to be added to the collection, and for preservation-, working- and access- duplicates. These form the basis of the Library's preservation policy for audio materials. They have also enabled effective quality control over outsourced preservation work.
From around 1992 most acquisitions were recorded directly to Digital Audio Tape (DAT) at 48 kHz, 16 bit linear. Though excellent for acquisition, and possessing very high reproduction quality at the time of recording, DAT is suspect as anything more than a short-term storage medium, and we have continued to rely on analogue tape for preservation and medium-term storage. However, the prospect of waning commercial support for professional analogue tape and related equipment forced us to start looking seriously at options for digital archiving in 1994.
The limitations of the standard analogue method for preserving audio material have always been apparent. It can only be carried out in real time; it introduces noise and distortion - some of which is irreversible; and the whole process will have to be repeated within 50 years or so, when the archival carrier itself will be in danger of degradation (as much of the earlier parts of the collection are today). To preserve the Library's digital (DAT) original by analogue copying means converting from digital to analogue, a process that introduces distortion, and recording onto a magnetic tape that introduces noise. This negates the advantage of digital audio recording. It is also extremely unlikely that an analogue tape player will be commercially available in 50 years.
These factors have caused a shift in electronic archives thinking. The sound archivist no longer focuses on extending the life of the carrier as far as possible. Instead, the archivist aims to select a digital standard that preserves the integrity of the recorded material, and a carrier whose archival life will exceed its projected obsolescence. As the recopying period is much shorter, the rate of the migration of the digital audio and the costs to do it are major issues.
When we started our investigations we expected to end up with a digital mass storage system - something that would hold all the data in a black box and do everything we could ever want. (We thought we were looking for that kind of solution - I believe it's called Digital Heaven.) The more we looked into it the less feasible the black box idea became.
It also became apparent that at this stage there is no single apparent successor to the analogue reel tape, and that for some time to come changing formats and shifting standards are likely to be the only predictable factor.
When we considered converting the collection to digital format we had to face the fact that no-one knew very much about the size of the task. A detailed collection information system simply did not exist. This made it difficult to make good decisions about the scope of the digital conversion, and impossible to allocate reliable priorities. To provide effective management of the collection, and to guide the conversion process, a comprehensive, searchable information system was needed. The Library made a significant investment in designing such a database and collecting data, taking every tape off the shelves and recording basic information about its make up, age, intellectual and physical control, transcription status, access restrictions, and preservation status and needs. The database provides information to support detailed priority setting and planning of preservation, cataloguing, transcription and collection management work.
At a collection-wide level, for example, the database shows that we have about 29,000 original recordings, covering about 30,000 hours of original recorded sound. We now know how much, and what, material has been copied to preservation standards (28%), how much has been fully catalogued (32%), how much transcribed (25%). We also know which items are on unstable types of tapes and warrant a high priority for action.
Library staff spent roughly two years in detailed investigation of the technical options for digitising the Oral History collections. This required consultation with local and overseas institutions, discussion with equipment suppliers and users, and in-house evaluation of options, advantages and disadvantages.
We came up with a list of at least 40 factors we wanted to take into account in making a decision. Critical criteria included things like the expected life of the technology and the support it was likely to get from suppliers; the amount of change that would be needed in our facilities, expertise and procedures; the down-time involved in getting a new system working efficiently and effectively; maintenance requirements for the new media; the availability of standards. And, of course, budget realities.
We were particularly interested in building in flexibility that would allow us to adapt to the inevitable development of new technologies and obsolescence of whatever system we would choose. The number of factors to consider, the different weighting of criteria, and the uncertain answers to many of our questions, all made decision-making complex. We found we could be more rigorous with some of the criteria and more intuitive with others. We recognised that it would be pointless to wait for the perfect, proven and stable technology to arrive, but that we would also be foolish to pioneer an attractive but untried and unsupported technology.
These investigations led to a bridging strategy that can be looked at from two perspectives - that of the broad technology chosen, and that of the specific program we set up to introduce the technology into the Library.
When we looked at all the shifting technological options we recognised we would not be able to move directly to a mass storage system - digital heaven had not arrived, at least within budget realities. We decided we would be best to look for a widely used, flexible technology that would allow us to record digitally and give us better archiving potential than DAT, while allowing us to migrate to some future digital technology in a fairly automatic manner. That way we would at least be introducing a bridge to a future mass storage system.
We chose Recordable CD (CD-R) - as the most available technology to achieve this. CD-R is not the ultimate solution to digital archiving, and many other technologies offer some specific advantages over it. Overall, however, it appeared to be the broadly acceptable solution to the Library's requirements for the foreseeable future.
CD technology has become so widespread, in so many different forms, that it will take a very powerful technology to replace it. That will surely happen, but we believe we have maybe another 10-15 years before CD technology is superseded. That should be less than the life of the media itself, so we should not have the costs of continual re-copying that would apply to, say, DAT, during the life of the technology.
There have been many questions about the long term permanence of CDs, largely stemming from experiences with cheap, poor quality pressed CDs, or from early forms of CD-R that used dyes that were vulnerable to UV. CD-R manufacturers now offer much more stable dyes that should do the job for at least as long as we will need them.
For the Library's purposes, CD-R offers a number of technical advantages, including the following:
There are some technical disadvantages:
The system we have chosen is basically a network of digital audio workstations (DAWs) with CD-R writing capabilities. The Sonic Solutions system we have installed consists of 3 networked ports in a hub of 8 possible ports, with direct recording or uploading to computer hard disk, management and editing of the recording, indexing, incorporating metadata, and so on, on-screen and then writing to CD, while simultaneously producing analogue reel to reel and access cassette copies.
Installation commenced in March this year and the system was fully productive by July. The Library's main contractor has also installed a similar system so that work sent to contract can be handled in the same way.
To minimise the failures associated with writing CD-Rs, to verify recording levels, and to ensure that the indexing points are placed correctly, the audio material is first assembled on a hard disk-based DAW. The CD is then written at real or double time on a SCSI writer in the background while the operator starts recording and assembling material for the next CD-R.
CD-R quality will be monitored with a CD tester and the relevant data relating to each disk will be stored. Longer term testing of the collection will allow degradation of the carriers to be tracked.
The change to digital technology has resulted in the need to re-evaluate previous archiving standards, procedures and terminology. In the analogue domain the production of material for the collection followed procedures accepted generally in Australia and overseas. An original recording made for the collection was recorded on a stable analogue tape. If an old and potentially unstable recording was acquired, then a stable preservation duplicate was made of it. In either case, a working reel on a different variety of tape and an access cassette were made from the original. The process was predicated on extending the life of the carrier and minimising the generations of copies needed (in order to save money and to minimise the impact of signal degradation associated with analogue copying).
With digital technology we have changed the procedure slightly but significantly - and more significant changes are likely to follow. We still make an analogue preservation copy on a stable tape. This copy may be better thought of as a backup or safety copy against the possibility of something happening with the CD, or against CD technology turning out to be a failure. We will almost certainly stop using analogue tape within a few years and make only digital copies, but for now we are still using an analogue preservation master.
Recordings - either from the studio, from field recordings, or from existing analogue masters, are also written to CD-R. This CD-R copy does a number of things: it takes the place of the second analogue copy we previously made as a working copy; when it comes time to migrate the data to another digital format, we will use the CD-R as the data source, for higher speed migration, so it is in that way also a preservation copy. It may also be used as our access copy eventually, within a secure networked environment, although for now we are still making access cassettes.
There are still uncertainties to be dealt with. For example:
These challenges in sound archiving are not happening in isolation from the rest of the Library. Developing long term strategy is becoming increasingly difficult, and we are all finding it necessary to build today on the best foundations we can establish, while keeping options open to help us adapt to what will be needed tomorrow.
| About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us |