Staff Papers
Physical format electronic publications in the National Library of Australia: report on a preservation survey
Paper by Deborah Woodyard, Electronic Preservation Section and Colin Webb, Manager, Information Preservation.
1. Introduction
The National Library of Australia has been collecting Australian electronic publications in physical formats for some 15 years - the first floppy disks entered the collections in about 1983. There has been some active management of their acquisition, cataloguing, storage, and access over the years. A decision was made in the early 1990s to rationalise their storage by creating special runs for them, deferring further decisions on their long term preservation until later.
Early in 1996 the Library decided to review the arrangements under which floppy disks and CD-ROMs are managed, especially with a view to long term accessibility. As part of this review the Library undertook a survey of its holdings of what we have variously called physical format, tangible, hand held, or portable electronic publications. This paper presents a report on the survey - its objectives, some of the approaches and assumptions behind it, the methodology used, and what we found.
It should be noted that the survey focussed on floppy disks and CD-ROMs only in the Library's general Australian collections. Online networked publications were not included, and even among physical format publications a range of floppy disks and CD-ROMs were not covered by the survey as a result of a series of pragmatic decisions. These are described under Methodology in Section 4.
2. Objectives of the survey
The underlying objective of the survey was to provide information that would help us manage the preservation of Australian information already held or entering our collections in these formats.
We recognised that long term this almost certainly means transferring data from the original medium, so ultimately we are mainly interested in the information rather than the carrier. However, the current carrier has many characteristics that may influence our ability to transfer the data.
Managing for long term accessibility is made up of many steps. In a complex library environment it is not simply a matter of focusing on an arbitrarily defined act called "preservation". Virtually everything that happens to this kind of material from the time it is created has a profound impact on its long term accessibility. Many of these influences affect the medium or the information directly - for example, the conditions in which it is stored, the way it is handled, the effectiveness of security arrangements. However, we also recognised that many factors having a trivial impact on carrier longevity may significantly influence our ability to manage the survival of the information. If we cannot find, operate and transfer this kind of material easily, it may in effect be useless even while remaining in a good physical condition.
In practical terms this meant the survey was designed to serve two primary purposes: to gather information on the nature and scope of the collection and of the task to manage it, and to assess our existing arrangements.
Preservation of this material is difficult for a range of reasons. It is very likely that the Library will not be able to maintain long term access to all aspects of all items, because of technological difficulties, copyright and licensing constraints and resourcing issues. We want to be able to use some of the information from this and other, more sophisticated surveys to help us set priorities for our own preservation action and as a basis for discussing cooperative action in partnership with others.
3. Approaches and assumptions - what were we looking for?
3.1 The size and the nature of the collection and of the management task
We wanted to know:
- how many publications were held outside the Special Collections, and how many disks (of some kind) were involved
- how many were CD-ROMs and how many were floppy disks, because we assume they will require different preservation responses
- the numbers of different sizes of floppy disk: we have limited facilities for anything other than 3 1/2" disks, so we needed to understand the scope of that most basic of hardware issues: disk drive requirements
- the spread of broad operating systems required for the items held: we know we have problems supplying more than the most common current operating systems.
- the age of material. We assumed that older floppy disks were likely to be most at risk of media instability, increasing error rates, and technological obsolescence
- what kind of content was held
- what kind of material accompanied the disks, if any
- whether we could establish a publishers' profile that might be useful for discussing ways of maintaining accessibility or influencing the way publications in the future are acquired and managed.
3.2 Existing arrangements for managing this material and how effectively it might facilitate long term preservation management
The Library was not starting from scratch in managing this material: there were a range of policies, procedures and practices in place to deal with acquisition, control and storage. We wanted to see how well they would support preservation action, and whether there were changes we needed to recommend and negotiate.
To address this we wanted to know:
- out of the items catalogued as having disks, how many were missing
- how well our documentation systems give us information that will help us find and manage this material. Critical issues for preservation are the ability to locate items using a limited number of search terms, and the ability to make basic distinctions such as the size and number of disks within a title, without having to go to the shelf or create a separate database
- where the material is stored. As already noted the Library established a policy of storing physical format electronic publications in separate runs with mixed media kits. We wanted to see how consistently these arrangements had been applied, and look for evidence that separate storage would provide better security than interfiling in general runs.
4. Methodology
Before we could conduct a survey we had to find the best way of locating material in the collections. The Library's Integrated Library Management System (ILMS) appeared to be the best way of doing this, as the most comprehensive collection-wide documentation system.
This survey concentrated on Australian materials in the general collections as identified by searches made on the ILMS in March 1996. Material added in the past 18 months is currently being surveyed and is not included in this report.
The searches were based on four search terms:
- "disk" in the collation field
- "disc" in the collation field
- General Materials Designator of "computer file"
- General Materials Designator of "interactive multimedia".
The survey was not based on sampling methods. We wanted collection-wide information that would support management decision making, but we also wanted the survey to guide our planning for specific items, so we tried to look at all known relevant material held by the Library, except for the categories of material listed below as exclusions.
4.1 Exclusions
As already noted, the survey focused on published floppy disks and CD-ROMs only. It did not look at the following:
- · online networked publications, (addressed in the Library's PANDORA project, described at URL http://www.nla.gov.au/policy/pandje97.html).
- recordable CDs (CD-R) produced by the Library in its digital
conversion projects (such as:
- - the Oral History digitisation program, described at URL http://www.nla.gov.au/nla/staffpaper/cwebb4.html ;
- - the Images 1 program described at URL http://www2.nla.gov.au/imagecoll/about.html;
- - the Australian Cooperative Digitisation Project 1840-1845 described at URL http://www.nla.gov.au/ferg/ ).
- audio CDs appearing in the general collections
- floppy disks or CD-ROMs in the Special Collections, even though we know a number are held in Manuscripts, Maps and Oral History. This was a purely pragmatic decision: we wanted to start with what we thought at the time would be more straightforward material, moving on to Special Collection material when we had some procedures in place
- updatable material in the loose-leaf and reference collections identified as intended mainly for current reference purposes and not for long term retention
- CD-ROM and floppy disk items which were not picked up by the catalogue searches. These were assumed to be mostly irregular supplements to print serials. The Library decided that it was likely that the value of this material would not justify the very considerable resources needed to locate and survey it. However, it is recognised that a small amount of valuable material may have been missed because of this decision.
5. Findings
(In the findings presented below, apparent discrepancies in totals are usually due to the overlap between different categories.)
5.1 The size and nature of the collection and of the management task
5.1.1 Number of titles and number of disks
There are survey results for 469 items, however it was discovered that only 405 of these actually contained computer disks. (See 5.2.1 for explanation)
Many titles included more than one disk. The total number of disks was 855.
5.1.2 Breakdown by type of disk
The numbers of CD-ROMs and floppy disks compared as follows:
|
|
Number of titles |
Total number of disks |
|
CD-ROMs |
93 |
134 |
|
floppy disks |
319 |
721 |
There were 7 items which contained both CD-ROMs and floppy disks (all containing 3.5"disks; one also contained 5.25" disks). In most cases the floppy disks and CD-ROMs appeared to contain different information, both of which would need to be preserved.
These results suggest that at the time the survey was completed, the Library's backlog of material that may require preservation was heavily weighted towards more vulnerable floppy disks, suggesting it may be appropriate to give them some priority for ekdown by size of floppy disk
Among the floppy disks, the breakdown of disk sizes (and therefore disk drive requirements) was as follows:
|
Size |
Number of titles |
Total number of disks |
|
5.25" |
123 |
172 |
|
3.5" |
223 |
549 |
Of the 123 items containing 5.25" floppy disks, 28 also contain 3.5" disks. This overlap simplifies preservation for these titles as the two disk formats appeared to contain the same information, reducing the number of 5.25" disks needing to be dealt with.
However, these results confirmed that the Library would need to go beyond the hardware available in the Library in order to deal with significant numbers of the larger disks.
5.1.4 Breakdown by operating system required
A majority of material surveyed was designed to work on an IBM-compatible PC. Macintosh was the other most prevalent system.
|
Operating system/Hardware required |
number of items |
|
IBM-compatible PC only |
261 |
|
Macintosh only |
46 |
|
either PC or Mac |
37 (26 of which are on CD-ROM) |
There were 61 items - all older materials - which required other operating systems:
|
Operating system/Hardware required |
number of items |
|
Apple II, IIe, IIc, IIgs, II+ |
38 (all 5.25" disks) |
|
British Broadcasting Corporation |
11 |
|
Commodore 64 or 128 |
2 |
|
Apricot PC |
1 |
|
Microbee 64 |
1 |
|
Unknown |
8 |
The Library would need to go beyond currently available software to deal with these other operating systems.
5.1.5 Breakdown by age
The publications surveyed were published during 1983-1997, and were broken down into three year periods as follows:
|
Date shown on publication |
number of publications containing: |
||||
|
years |
5.25" |
3.5" |
CD-ROM |
|
|
|
1983-85 |
28 |
1 |
0 |
|
|
|
1986-88 |
32 |
13 |
0 |
|
|
|
1989-91 |
34 |
22 |
0 |
|
|
|
1992-94 |
26 |
107 |
36 |
|
|
|
1995-97 |
0 |
>67 |
>54 |
|
|
|
span of production |
1983-1994 |
1985-present |
1992-present |
|
|
This indicates that a substantial body of vulnerable floppy disk material is more than 5 years old. It also confirms the impression that there has been a substantial change in the kind of formats needing to be dealt with and that the nature of the technical problems will change over time.
5.1.6.Content
We attempted to categorise the type of content held on disk, using a rough content classification system devised by Preservation Services for the survey.
|
Content category |
% of titles |
|
educational - up to secondary |
20 |
|
computer instruction |
20 |
|
educational - tertiary |
10 |
|
reference (eg. bibliographies, encyclopedia, genealogy) |
13 |
|
popular culture, hobby subjects, etc. |
9 |
|
software (ie. a tool, not carrying unique information) |
6 |
|
government publishing |
5 |
|
Australian culture and history |
5 |
|
instructional (informal teaching, do it yourself material) |
5 |
|
scholarly papers, conference proceedings |
4 |
|
literature |
2 |
|
game (other than educational for schools) |
1 |
As the table shows, the most common material surveyed could be characterised as educational material or computer instruction.
We recognise that these categories are ambiguous and overlap each other. More rigorous use of CDP to define what should be preserved will require consideration of questions such as:
- whether we can find unambiguous and useful categories for classifying content
- whether we should preserve these formats more selectively than for print formats
- how we can assess the content value if we need to.
The Library has drafted Acquisition guidelines for these formats. These guidelines indicate a more selective acquisition policy than for print, but probably a broader acquisition outcome than we will be able to preserve. We still need to find meaningful criteria for selecting material for preservation, recognising that in this environment a low preservation priority probably means an early loss of accessibility.
5.1.7 Accompanying material
Most disks were accompanied by some form of printed text.
There were 128 items with accompanying pamphlets containing only information on how to load and run the computer files on the disk(s). Another 268 items were accompanied by more substantial books containing information beyond that required to operate the disk(s), and likely to be of value in its own right.
These findings raised a question about print equivalence: the extent of overlap between the electronic format and the print material. This was not assessed throughout the survey because of the time that would have been required to do it.
Of the 64 electronic publications for which an assessment of print equivalence was recorded, 9 items were judged to have parallel versions in print.
5.1.8 Publishers' profile
The most common publishers and their period of publication activity were:
|
Publisher |
general type of publications |
number of items recognised |
typical format |
period |
|
Informit/RMIT |
various |
14 |
PC CD-ROM |
1994-96 |
|
McGraw Hill |
computer, educational |
14 |
PC 5.25" and 3.5" |
1985-97 |
|
Butterworths |
reference |
13 |
PC or Mac, 3.5" and CD-ROM |
1994-97 |
|
Nene Valley Press |
computer |
12 |
PC 3.5" |
1991-95 |
|
Data Publishing |
computer |
12 |
PC 3.5" |
1989-97 |
|
Prentice Hall |
computer |
11 |
PC 3.5" |
1990-95 |
|
Webster publishing |
popular |
9 |
multimedia PC CD-ROM |
1994-96 |
|
CSIRO |
software, popular, instructional / educational |
9 |
various |
1988-96 |
|
ALIA |
reference |
8 |
PC 3.5" |
1993-95 |
|
Education Dept Tasmania |
educational |
8 |
BBC |
1986-88 |
|
AGPS |
government |
7 |
PC 5.25" and 3.5" |
1987-95 |
|
Dataworks |
popular |
6 |
Mac 3.5" |
1992-95 |
|
Australian Bureau of Statistics |
reference, government |
6 |
PC various |
1994-96 |
|
Mitchell CAE |
cultural, educational |
5 |
Apple 5.25" |
1985-88 |
This seems to suggest that it may be possible to achieve much by identifying a relatively small group of publishers with whom to discuss ways of preserving their publications. However, it is sobering to realise that this list of publishers with five or more titles identified, accounts for less than one third of the publications surveyed: only 134 out of approximately 469. As well as these higher output publishers, the market seems to have been characterised by a great number of smaller publishers.
5.2 Existing arrangements for managing this material and how effectively they may facilitate long term preservation management
5.2.1 Match between catalogue entry and availability of disks
As already noted, the survey addressed 469 titles recorded on the catalogue; only 405 of these were found to contain computer disks when the item locations were checked. Of those that didn't have disks:
- 23 had no part of the item on the shelf
- an additional 41 had no disks, even though other parts of the item were found.
The apparent loss of disks could be attributable to a number causes: disks never having been supplied; mis-shelving; theft or loss of the disks; or misinformation derived from copy cataloguing.
5.2.2 Catalogue information about size and number of
disks
The collation field in the ILMS record for 400 items was
checked for the size and number of disks - basic information required for
preservation management. Only 238 gave complete details.
This information is not required under existing cataloguing rules for disks accompanying print materials. Following a request from Preservation Services that information is now routinely recorded in the catalogue entry for monographs. The results recorded in this survey do not reflect those changed procedures because the material surveyed was catalogued prior to the request being made.
The results suggest that preservation managers may not be able to rely on the catalogue record to provide some of the basic information they need, unless they can negotiate local changes as we have been able to do at NLA. Even then there may be problems when copy cataloguing data is used.
This has highlighted the gap that may exist between information needed for current bibliographic access and that needed for long term preservation management, raising questions about how and where the latter should be recorded. For example, the Library has decided to incorporate this information in a special metadata archive being developed for online publications in the PANDORA project.
5.2.3 Storage arrangements
Looking at where the surveyed material was shelved (and bearing in mind that Special Collection material was not included), we found that:
- approximately 70% of the material surveyed was held in the special mixed media runs (292 items in the monograph run; 44 items in the serials run)
- approximately 30% was in the regular Australian runs (133 items).
Although one would not want to read too much into this, it is interesting to note that the survey showed:
- approximately 37% of the items in the regular runs were missing completely or lacked disks which the catalogue record said should be there
- only around 4.5% of the items in the special runs were missing completely or lacked disks which the catalogue record said should be there.
In some cases these apparent losses may be due to disks not having been provided with publications that were copy catalogued. However, it is also possible that the storage locations were significant:
- storage containers are more consistently supplied for items in the special runs, so there is less chance of disks slipping out, and
- the use of ProLoc cases and specific access procedures for special run items may provide better security than in the regular runs.
One special aspect of storage is the way disks are labelled. We did not specifically record information on this in the survey, but noted that many of the floppy disks were not labelled with the item call number. Some disks carried no label of any kind. This becomes a serious problem if the disk is for some reason separated from its accompanying material which may be labelled. We assume that accurate labelling of some kind maybe critically important for security and for reshelving misplaced items.
6. Follow up
We are currently completing a survey of more recent acquisitions. The results of these surveys are helping us look at the scope of the demands on our resources, which in turn will influence our planning. We are also undertaking migration trials for a range of material identified in the survey, to establish useful pathways for preservation.
We are very interested in discussing the implications of such surveys, both here and in other institutions. Comments can be addressed to Colin Webb, Manager, Information Preservation, email cwebb@nla.gov.au or Deborah Woodyard, Electronic Preservation, email dwoodyar@nla.gov.au .
