When Works does not work
A journey from convicts to digital preservation
12 August 2016
The following true story nicely illustrates what digital preservation is about and why it is important.
Some time ago, a user in our Main Reading Room requested an item from our collection titled ‘The Complete First Fleet’. This database of all 1,441 people known to have been part of the First Fleet published in 1992 is an example of our early digital collections.
The goal of digital preservation is to provide access to the intellectual content of digital objects in the long term. By ‘long-term’ we mean decades and possibly centuries. From this perspective, The Complete First Fleet represents the most critical issues we encounter in this endeavour.
Firstly, the database originally came on a 3 1/2 inch floppy disk. Since then, Floppy disks have been long replaced by CD-ROMs and USB Flash drives. Modern computers, including the computers in our reading rooms, do not have the necessary drives which would allow reading floppy disks.
In addition, physical carriers such as floppy disks or CD-ROMs are prone to degradation and have a limited lifetime. Depending on quality of the media and the way they are handled, this can be as short as few years.
Luckily, we were forward looking and in 2008 built a system called Prometheus which copies content from physical carriers and stores it in the managed mass storage system called DOSS. This removes the dependency on the fragile and obsolete carriers. Processing content published on physical carriers through Prometheus is now a business-as-usual task performed routinely by our cataloguers.
So, the content of The Complete First Fleet can be removed from the physical carrier, which means we no longer need to worry about having the right hardware to read it, it can be safely stored in DOSS and properly replicated—this sounds like a happy ending and the end of the story, right?
Well, no—in fact, this is only where the real story of digital preservation begins.
What we have done here is made sure that we have kept the original bit stream, the 0s and 1s that constitute the digital object. This is a necessary but not sufficient condition for maintaining access to the intellectual content of the object. Unlike a book, the digital object cannot be directly opened and read by a human—it is only possible through intermediation by a machine which will decode the content and present it to the user in an understandable form.
The next logical question is then ‘can current computers support access to the intellectual content of The First Fleet?’ When we tried opening the file, Windows 7 which is the current standard operating system used in the Library complained that it did not know what format it was. Given that we know that the format is Microsoft Works database (.wdb), the fact that Microsoft Windows do not recognise it gives us an indication that the format is no longer supported.
Indeed, we know that Microsoft Works was discontinued after version 9 in 2007. No other current application including Microsoft Access and OpenOffice Base can open .wdb files.
So, is there anything we can do? Well, we can try to get a copy of the Microsoft Works application. The Digital Preservation team actually have an extensive library of old software applications and we happen to have two versions of Microsoft Works there—version 8 and 4.5.
We first installed the newer version 8 in an emulated environment and tried to open the file—unfortunately version 8 turned out to be backward incompatible and could not open the file created in 1992. We then did the same with version 4.5—and this time we succeeded.
From here, we have two options for enabling access to the collection item for our users:
Firstly, we could consider installing Microsoft Works on the computers in the reading room. It actually turns out that the application can run on current versions of Windows which means we would not need to emulate an older operating system—at least not in the immediate future. However, it will very likely be necessary in some time. This option would also have legal implications as we would need an appropriate license to run the software on a number of computers. And lastly, the backward incompatibility means we would have to think about running different versions of the software to support multiple file format versions.
For these reasons, we have decided to explore the second option which is to transform, or in digital preservation lingo to migrate, the file from the Microsoft Works .wdb format into another one which is supported by the current standard operating environment. We have migrated the .wdb file into DBase 4 (.dbf) format which can be opened by Microsoft Excel and then saved it as Excel's .xls format which is well supported by all our computers including the ones in the reading rooms.
Stories such as this one highlight the need for digital preservation as a robust, managed process. We have only discovered that The Complete First Fleet was not accessible by coincidence, because somebody requested it. We then took a laborious, hand-crafted action to remedy the problem.
Although it worked and fixed the problem, it would not scale.
We now have billions of digital objects in our collection. How many more items like the Complete First Fleet are there? We cannot wait for each of them to be requested by a user just to find out that there is a problem and that the item cannot be accessed. We need to know that there is a problem ideally before the item is requested so that it can be fixed. Knowing about a problem early can also significantly reduce cost of any follow-up action required and at the same time increase the likelihood that it will succeed.
This means we need to be proactive and have scalable processes for maintaining the long-term access to our collections. We need a digital preservation system which will allow us to understand what we have in our collection, proactively monitor the digital material for any potential problems and then take appropriate actions, both on small and large scales.