Paper by Deborah Woodyard, Electronic Preservation, National Library of Australia
Practical strategies for long term access to publications on computer disk are being developed by the National Library of Australia. A survey of general collection material on disk was conducted and practical aspects of migration of floppy disks to CD-R have been tested and are reported in detail. Integral to the benefit of this work is cooperation with State Libraries to develop a national strategy for the preservation of electronic publications in physical formats. This topic was the subject of a special meeting with publishers and State Libraries on October 27 and 28, 1997.
The National Library of Australia has been collecting Australian electronic publications in physical formats for some 15 years - the first floppy disks entered the collections in about 1983. There has been some active management of their acquisition, cataloguing, storage, and access over the years. A decision was made in the early 1990s to rationalise their storage by creating special runs for them, deferring further decisions on their long term preservation until later. Early in 1996 the Library decided to review the arrangements under which floppy disks and CD-ROMs are managed, especially with a view to long term accessibility.
Managing for long term accessibility is made up of many steps. In a complex library environment it is not simply a matter of focusing on an arbitrarily defined act called "preservation". Virtually everything that happens to this kind of material from the time it is created has a profound impact on its long term accessibility. Many of these influences affect the medium or the information directly - for example, the conditions in which it is stored, the way it is handled, the effectiveness of security arrangements. However, we also recognised that many factors having a trivial impact on carrier longevity may significantly influence our ability to manage the survival of the information. If we cannot find, operate and transfer this kind of material easily, it may in effect be useless even while remaining in a good physical condition.
The discussion paper entitled National Strategy for Provision of Access to Australian Electronic Publications (1) has confirmed the commitment of the National Library to providing long-term access to publications in electronic formats, both networked and in physical formats. It indicates that the Library will initiate and participate actively in national discussions and research in the management and preservation of these materials.
This paper will refer to two particular activities which illustrate the Library's commitment in the area of publications entering the collections as floppy disks and CD-ROMs. The first is a case study of the research undertaken at the NLA on the migration of material from floppy disks to CD-R. Secondly, the results of this research and other NLA activities formed the basis for a national meeting held on October 27 and 28, 1997 and I will be highlighting in my talk the steps taken at this meeting towards the development of a national management and preservation model for physical format electronic publications.
Issues such as media instability and technological obsolescence are threatening to greatly reduce the longevity of these publications unless they are carefully managed.
Preservation includes all the actions taken to extend the useful life of these materials and particularly the information contained on them. Therefore control, storage, handling and security are involved, as well as copying and reformatting or migration. Overall this is a very complicated process and many issues must be addressed before libraries can effectively preserve this material, including:
Management Issues
Generic Preservation Issues
Media Deterioration Issues
Technological Obsolescence Issues
New Preservation Pathways
CD-ROMs are still a current and popular technology with relatively stable media, so the National Library of Australia has first focussed attention on the floppy disks in the collection. It has been agreed within the Library that copying and migration of information on computer disks is likely to be the most useful preservation method currently available to us. In order to examine how we might implement these strategies a series of trials will be conducted. The results of the first trial are described below.
The overall aim of this project was to migrate Australian publications that exist on floppy disks to CD-R (recordable compact disc).
It became clear that this was more an exercise in copying data from one medium to another, which is only a part of the process of migration, rather than completely addressing the problem of changing technology, such as the operating system or associated software dependencies. This has a valid purpose, considering the average life of a floppy disk is estimated at less than ten years.
Identifying and Locating Disks
The ILMS (Integrated Library Management System) was recognised as the most useful source of information for identifying and locating disks in the collections. Searches were made for material with:
This produced a total of 1 439 records and appears to have captured a majority but not all items. The only serials located in this search were those where the primary material was a computer file. Print serials with accompanying disks were not located. Many of these records however were overseas publications which are generally outside our selection guidelines for preservation.
It has become apparent that more information than is currently available in catalogue records must be recorded for the successful preservation management of this material.
Planning has commenced on including the extra metadata and information required for these physical format electronic publications in the PANDORA database which is being created primarily for online publications. To help simplify the management of databases and metadata in the NLA it makes sense to utilise the same database for all electronic collection materials.
Selecting a Representative Sample
In order to determine the various complications that may be encountered in transferring publications to CD-R a representative sample of material had to be chosen for testing. A survey (2) was carried out of all Australian material located in the general collection to determine the range of material to be considered.
The survey covered 469 publications, 405 of which contained computer disks, and 64 of these were chosen as a representative sample of material containing floppy disks. The sample of publications were chosen to represent a number of variables:
Copying
Basically the copying process devised was to transfer the data from the original floppy disks to a network drive, download the data to the hard drive of the PC containing disk writing software and then write to CD-R. The entire process (with the exception of file comparison) was carried out with hardware and software currently available in the Library.
A detailed description of the copying process is given below.
a. Select appropriate computer hardware and operating system, ie PC vs Mac, with correct size of floppy drive.
23 items could not be used at all due to a lack of the appropriate hardware or software. Another 1 item could not be used because a security tag had been stuck over the metal sliding cover on a 3 1/2" disk. This reduced the number of materials to continue through the rest of the trial to 42.
b.Virus check of original floppy disk
All 42 items were checked for viruses using the latest version of virus scanning software in the Library. No viruses were found.
c.Physical disk scan of original floppy or similar process to check for media deterioration
Scandisk was used on PC material, and Nortons Utilities Disk Doctor was used on Mac material to examine the disks for errors and deterioration. Only 2 PC disks had problems. One disk had errors which resulted in it no longer being operational so it was taken off the list for preservation, but the other had a second copy and could still be used. This reduced the number of titles for the remainder of the trial to 41.
d.Install and run from floppy
At this stage of testing one publication was discovered to have a blank disk although it was labelled as containing the appropriate files. Therefore it was also eliminated from the rest of the trial, leaving the final 40 publications.
When trying to run the files further difficulties experienced include:
Twenty-seven items worked as they were supposed to when used from the original floppy disks.
e.Copy contents of floppy disk to network drive
This was successful for all 40 items.
f.Run comparison software to check for differences between original and copy
This was successful for all PC items. Mac items were not tested because appropriate software for the task could not be located.
g.Set as Read Only
This was tested for PCs only. It was not until after this part of the trials was completed that it was confirmed that "locking" a file on a Mac has the same effect as "read only" on a PC. This was mainly to test for the ability of the files to continue to function as required in a read only state as they would exist on the CD-R.
h.Install and run from network drive
Running files that had been set as read only did not cause any problems for these items.
All items continued to function after having been put on the network drive, although 4 items had to be copied back from the network to the root directory of a floppy in A: drive, and one of these required the renaming of the floppy disk, before these copies would work. This is because the expected location for the files, A: drive, has been hard-coded into the item.
The amount of time required to process material to this point in the copying procedure (ie steps a-h) was measured for 18 items, and took an average of 40 minutes per item.
i.Unset Read Only
Files that had been changed to read only status were returned to their original state so that they would still represent the original as faithfully as possible.
j.Organise titles in appropriate file structure using appropriate naming convention.
In an effort to apply widely accepted standards to promote longevity the data was arranged into a file structure that applied the ISO 9660 standard (3). This entailed using only A-Z (all capitals), 0-9 and the underscore in the 8 character plus 3 character extension naming convention for creating the directory structure for storage. The depth of the directories can reach a maximum of 8 levels.
The original file names for many publications however introduced illegal characters and extended names which could not be changed because their own programs would not recognise the new names and would cease to operate. Mac files in particular did not follow the basic ISO 9660 naming convention.
k.Create and include documents with appropriate details such as installation instructions for each item.
A general file was created in ASCII text to describe the contents of the CD-R, how files are arranged on it and refers the user to individual ASCII text files for instructions on how to use the individual items.
For this trial the individual ASCII text documents were created for only 4 items to test whether this is a successful method for carrying important information about each publication. In practice there would be one file each created for every item on the CD-R.
Apart from obvious details such as title, the ASCII files for the individual items include:
These in future should also include date the disk(s) was copied to a new format and what that format is. This may seem obvious now, but if this data is migrated several times it may be desirable to have the history recorded.
l. Write to CD-R
This involved copying the files from the network drive to a computer set up with the software for creating CDs and a CD writer. During the process this software checked the file structure against the ISO 9660 file naming standard with Joliet extensions to allow for larger file names, eg for Windows 95. Four file names were rejected for containing illegal characters and had to be changed before the writing could continue. The rejected file names were all from Mac files which contained illegal characters according to the standard and extensions. This would most likely render the publications unusable.
Writing the 40 items to one CD-R (approximately 1,900 files, a total of around 164Mb) took 45 minutes.
m.Run comparison software to check for differences between original and CD-R copy.
Material for PCs compared favourably between the original and CD-R versions. Mac material could not be tested due to lack of comparison software.
n.Install and run from CD-R
When the CD-R was used to install and run the material the PC files did not appear to have any further problems, but it became apparent that the Mac files did not work.
The final result out of 40 items:
Further investigation has shown that Mac files need to be written to a CD-R using the HFS file system, which will require different CD writing software.
This also suggests that copying Mac and PC files separately onto separate CD-Rs would be simplest.
Research Conclusion
The testing has produced useful results. Not all the material in the sample was successfully migrated, however this was expected. We now have more informed views of the issues involved, the hardware and software required by the collection for use and whether it is available in the Library, and also the copying processes.
Further Action for Internal Consideration Flowing From this Trial
This trial highlighted the need for the Library to:
The results of the trials just described were presented to the participants of the national meeting in October 1997. This meeting was coordinated by NIAC (the National Initiatives And Coordination Branch at the NLA) and involved the State Libraries, National Library and invited publishers in discussion of issues relating specifically to floppy disks and CD-ROMs in deposit libraries.
The aims of the meeting were to:
More recent details can be obtained by viewing this paper at the VALA 98 website.
Outcomes of the National Consultative Meeting 27-28 October 1997
Below is a summary of the main observations from the meeting:
The National Library will produce a more detailed position paper stating clearly defined aims and activities which will lead to the production of a draft national policy for electronic materials.
Report on the First National Consultative Meeting on the Management of Australian Physical format Digital Publications 27-28 October 1997
Overview of each librarys current strategic directions and digital initiatives
The answer has not been provided yet but these activities, the meeting and the migration trials, are all steps towards defining our roles and making progress in the preservation of electronic materials.
1National Strategy for Provision of Access to Australian Electronic Publications: A National Library of Australia Position Paper [online] (Canberra, Australia): National Library of Australia, 24 December 1996 [cited 10 October 1997].
2 WOODYARD Deborah, WEBB Colin. Physical format electronic publications in the National Library of Australia: report on a preservation survey [online] (Canberra, Australia): National Library of Australia, 9 September 1997 [cited 10 October 1997].
3 HARRIS James R. iso-9660 Simply Cds [online] [cited 10 October 1997]. Formerly available at: http://www.success.edu/~jharris/html/iso.html
| About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us |