Staff Papers
Future Access - Long Term Preservation of Australian Electronic Legal Publications
A paper prepared for ELI 97 - Standards and Issues in the Electronic Publication and Dissemination of Legal Information. College of Law (http://www.opennet.net.au/collaw/), Sydney, 5 April 1997 by Wendy Smith, Manager - Pandora Project.
CONTENTS
- Introduction
- Electronic Legal Publishing and its Prospects for Survival
- Print vs Electronic Publishing
- Physical Format vs Online Electronic Format
- Pandora
- Appendix
Abstract
This paper looks at some of the issues involved in ensuring long-term access to Australian electronic legal publications. It compares print to electronic formats, and notes some of the special characteristics of electronic formats. It describes the Pandora project, initiated by the National Library of Australia, to archive Australian electronic publications selected for national preservation.
Introduction
Information published in printed form in books, journals and newspapers is collected, catalogued and preserved by libraries throughout Australia, so that all Australians can have access to their published heritage. This is guaranteed by legislation, with publishers of printed materials like books, newspapers, journals and maps required to deposit copies of their works in the National Library of Australia and relevant state libraries.
Legal deposit is a special privilege enjoyed by only a small number of libraries in Australia. The existence of legal deposit provisions have allowed these libraries to build up valuable and comprehensive collections of historic Australian published materials which otherwise might have been discarded - perhaps as they lost their immediate appeal or became brittle or damaged.
Currently, no similar obligation exists for electronic materials. The long term availability to Australians of the full range of their published heritage will be at risk if electronic information is not collected comprehensively and maintained similarly to printed information. The National Library is committed to preserving selected electronic publications of lasting cultural value for access by the Australian community now and in the future, and has commenced a project which is investigating procedures for ensuring this. With electronic publishing in its infancy, the time is right to commence an 'archive' of electronic publications which will document these historic developments for future generations.
Electronic Legal Publishing and its Prospects for Survival
The legal profession in Australia has taken up the cause of electronic publishing with enthusiasm. More and more law related material is being made available in electronic format - either in 'hardcopy' like CD-ROM or floppy disc or online via the Internet or other networked service. The range of materials available is very large - from primary legal materials to commentaries and discussions - and the rate of growth shows no sign of slowing down. While some information more or less duplicates material already in print, some of it is only produced in electronic format. Three titles, which illustrate some of the characteristics of electronic information are discussed in the Appendix.
Electronic materials have certain characteristics that mean their survival prospects are far less assured than those of print media. For instance, early versions of CDROM formats came with statements that they'd last 'forever'. Even if this was true, there is no guarantee that the information contained in them would be accessible into the future. Firstly one would also need to maintain a CDROM player 'forever'. Considering that very few people have 78 rpm drives on their record players, (if they have a record player at all), how likely is it that they would have the correct equipment for their CDROM in 30 or 40 years time? Has anyone tried to access the information on a 5 1/4 inch floppy disc recently?
Even if they could, it is even less likely that they would have the software to read and interpret the digital signals encoded on the disc. At this stage, software obsolescence is running at about twice the rate of hardware obsolescence. At best a meaningless jumble of digital signals would be produced.
It is probably fair to say that if no active steps are taken to ensure access into the future, materials being produced in electronic format today will not be available to the next generation.
Publications where the information content has commercial value; are heavily used; and generate sufficient income for the publishers, are likely to be updated or 'migrated' as technology changes so that new 'editions' are always available. For freely available or public domain materials, this is less likely since the ongoing cost of maintaining material may outstrip any resources available to do it.
Even if a publication is updated, there is a question as to who maintains the previous 'editions'. Aunty Abha's Electronic Publishing makes the point that: With legal research, it is sometimes necessary to know what the law was a few months or years ago. Subscribers to Aunty Abha's CD-ROMs can keep all back copies for historical use.. If technology changes, any back copies may end up being useless unless they also are migrated to a new format.
The more likely prospect for many publications, however, is that each successive version overwrites the previous, or else the previous version is discarded when a new version is issued.
One solution to the problem of historic availability of information is to archive selected materials in a dedicated repository, while updating the information on an ongoing basis and migrating it as technology demands. This is the solution adopted by the National Library of Australia as part of its Pandora (Preserving and Accessing Networked Documentary Resources of Australia) project.
Print vs Electronic Information
To ensure their long-term preservation and access, a different set of procedures need to be put in place for electronic materials compared to those used for print. There are some important differences that need considering.
Lack of Legal Deposit Provisions
The Copyright Act of 1968 is currently under review, including its legal deposit provisions. At this stage, any project to archive electronic information with the aim of providing long-term access can only be with the permission of all owners of the information being captured.
Since 1994, the Library has adopted the approach of contacting Australian publishers of hardcopy electronic materials to seek the voluntary deposit of CDROMs and similar media. Many electronic publishers have already responded and publications covering a wide range of topics have been deposited. Deposit does not preclude the possibility of purchase of additional copies where high levels of use are anticipated.
The concept of deposit is harder to define with online electronic publications, and by their networked nature they can be made available to many users at any one time. Rather than ask for a copy of online material to be deposited, the Library is seeking the agreement of online publishers for permission to capture into the Library's own electronic archive a copy of the material as it exists on the Internet.
Fair Dealing and Provision of Access
Provision of access to print materials by libraries is well defined, with copying and fair dealing provisions reasonably straightforward. The same cannot be said for the electronic environment.
The Library realises that most 'hardcopy' electronic publications are expensive to produce and is careful to make sure it does not affect the commercial interests of the producers. The Library has drawn up a set of conditions to govern the use of deposited electronic publications in the Library and to prevent unauthorized copying. Access is generally limited to one user at a time within the Library building.
At present, there is little online commercial publishing on the Internet in Australia, and the Library has not yet attempted to archive commercial materials. We recognise, however, that commercial interests need to be considered and will attempt to establish archiving agreements with commercial publishers with this in mind. Possible options include the withholding of archived information for a set period, or the restriction of access to approved clients only. These models have been successfully implemented overseas.
For some materials which are freely available or in the public domain this will be less of a problem. Some organisations - eg AustLII , make it clear that they are happy to have their material made readily available
7.6) AustLII aims to put on the Net 'public legal information': primary legal materials (legislation and decisions of courts and tribunals); and secondary materials that are (or ought to be) in the public domain or able to be licensed for free. AustLII's public policy agenda is to convince governments, courts, law reform bodies and other publicly-funded organisations to make legal materials they control available free via the Internet.
AustLII is one of many organisations aiming to ensure that some part of cyberspace is public space, where no one is denied use of resources because of financial considerations. Its aim is to create a public law library on the Internet. (http://www.austlii.edu.au/austlii/faq.html - FAQ 7.6)
At this stage, all publishers of online materials selected for preservation by the National Library of Australia have willingly given approval for their material to be archived. No major commercial publisher has yet been approached, but we are hopeful that we will be able to reach mutually acceptable agreements with them too.
Value Added Information
Electronic publications are not the same as printed materials, even if the actual information contained in them is identical. They almost always have some important 'added value' features - such as online links to similar documents; keyword searching; or browsing facilities. Anyone used to regularly using electronic information would find it very tedious to go back to a print based system (would we want to swap an online library catalogue for a card catalogue?) Not only is it important to preserve the information, it is also important to preserve those value added features.
Hypertext links are a feature of publishing definitely not found in the print media. Some online titles consist almost entirely of links to other documents, which themselves are not necessarily published by the owner of the original title. Problems of copyright and of identifying ownership can become quite complex very quickly. If all the actual linked documents were included, then the size of one single title could become encyclopedic.
Life Expectancy
It is known that the printed word and even microfilm have the potential to last and still be accessible for hundreds of years. There are international standards for paper quality, for processing and storage of microfilm, and for many other factors that affect longevity. But how long are electronic materials going to last? And how long will one be able to access them?
Online electronic materials have no physical format to preserve. They can be removed from the Internet or overwritten almost instantaneously at any time. For material identified as nationally significant, there is an urgency to capture it close to its time of creation to ensure it survives for the future.
Potential lifetimes of some modern media- initially quoted in hundred's of years - are now thought to be rather optimistic. However, rather than being constrained by the actual physical survival of the object itself, there are other factors which are much more important when considering the useful life of electronic information.
Useful Life
With paper and even photographic materials, it is usually the physical object that fails first - paper becomes brittle, photographic images fade or get scratched or otherwise damaged. As long as the physical object lasts, however, it can still be used - even if only to be copied one last time onto a more stable medium. In addition, printed materials are eye legible - all a person needs is good eyesight (or a magnifying glass) and he can read anything published in print or microfilm.
It is not so simple with electronic materials. Not only do they need machines (the hardware) to 'read' the digital signals, they need some special coding (the software) to manipulate, interpret and convert the signals into a human intelligible format. Additionally, the value added features such as searching facilities that make electronic information so attractive are also software based. All the software and the hardware needs to be available if the 'look and feel' of the information is to be maintained.
Hardware and software are not universal - they are specific to each format of electronic material and each sort of code used. In fact, with hardcopy electronic materials, the physical object like the CDROM or floppy disc may still be in existence long after the machinery to play it or the software to interpret it have been scrapped or become obsolescent.(Although if a hardcopy electronic object is going to fail, it generally does so suddenly and without warning.).
Generally, two types of software are involved - the software that allows the machine to read the information and the software that interprets the information. It is no use just capturing the basic bit pattern of an electronic document. It has to remain a useable copy of the original document, and needs all the necessary software to ensure it.
How Much Can We Preserve and Make Accessible 'Forever'?
The relative ease of mounting an online electronic publication means that there is an ever escalating amount of material appearing in that format. There is so much available even now that organisations like the National Library of Australia, which have a role to preserve the printed literary output of Australia, are not able to apply the same collecting procedures to online electronic materials as they would to their print equivalents. At present, the costs and staff time involved in acquiring and making available this material are much greater than for its print equivalent, and it is hard to see how this will be reduced, at least in the short term. As a result, there is a need to be much more selective.
Although the volume of material in hardcopy format is probably not so great - partly because of the significant production costs involved - selectivity is likely to again be necessary. Updates and cumulations can rapidly increase the amount of material to be preserved.
With much online electronic information, the software needed to interpret it is easily identified. However, software coding for CDROM format is often proprietary, and may be embedded in the object itself. Password or other protection can also create problems of access. If hardcopy format electronic materials are to be archived, then full details of all their operating and interpretive software have to be known.
Physical Format vs Online Electronic Formats
There are some other differences between the various formats of electronic documents that will affect their long-term preservation.
Permanent Location
A hardcopy electronic object has a physical format and can be identified and located by traditional library based systems. For online electronic materials, there is no physical format. Information can disappear without trace at any time if a publisher ceases to maintain his site. Alternatively, it might become very difficult to find information if he changes his site without leaving a forwarding address. Any hypertext links to his site from other documents will also cease to be active.
This is not a trivial matter where online information is concerned. A National Library of Australia bibliography on online information, compiled in early January 1996 still points to the AustLII site at UTS, which it obviously left some months ago. Fortunately, an online forwarding address is still available at the original site. Out of 54 entries in the Library database, only 37 titles (or 68%) were still at the same location 12 months later - the rest had moved on or disappeared altogether.
There are some systems being developed that will give a measure of permanence or persistence to locations on the Internet - PURL's, URN's, handles etc. It is intended to incorporate a permanent naming system in the Pandora archive.
Stability of Information - Currency, Authenticity
Once a CDROM is mastered, its information content is fixed. However, for online information it is relatively easy for the publisher or other person to change or edit it online. In quoting online electronic information, not only is the date of creation important, but also the date of viewing (or downloading).
One bonus of the relative ease of changing material online is that it gives publishers a flexibility they never had in print. Corrections, updates, reader feedback, reviews can all be included as they come to hand. This means that the concept of the integrity of a single print 'edition' can completely disappear in the online version. This can make the decision as to when or which 'copy' to preserve very problematical
AustLII discusses the concept of authenticity :
6.3) How do I know that somebody hasn't broken in and changed the law?
We are probably far more concerned about this than you are! One of our main system administration tasks is the maintenance of a fairly tight hold over security. It is for this reason that our services are split between multiple services, and also why some services are simply not available from the AustLII sites. (http://www.austlii.edu.au/austlii/faq.html FAQ 6.3)
Similar concerns of authenticity will also need to be taken into account in an online archive.
Finding Online Materials - Metadata
One of the problems of searching online for information is to find a way to restrict the amount of materials retrieved through popular search engines like AltaVista and Lycos. Because of the methodology of their search strategies they pick up everything containing your search terms - the important, the serious and the trivial - as they search the various indexes they use. These indexes can be built from a variety of sources, some from the titles of documents found on the Internet, some from the documents themselves and some from other indexes or directories.
Although there have been considerable improvements in their design, search engines cannot always closely interpret a user request. For example, trying for 'Australian constitution' on AltaVista got over 300 000 matches. Matches 1 to 10 were directly relevant. Match 11, however, was the Institute of Australian Geographers Constitution. Using the Aussie Web Wombat, which is Australian specific, there were only 1400 matches, but match 1 was for the Constitution of the Lyrup Village Association, South Australia. The Australian Constitution did not appear till match 19.
There are emerging international standards like the Dublin Core for recording metadata (sometimes called 'information about information') for Internet documents. use of these should improve the 'hit rate' for documents by using standard searching techniques.
Pandora (Preserving and Accessing Networked DOcumentary Resources of Australia)
This project aims to select online electronic titles for preservation by the National Library of Australia and to implement procedures to archive and make it accessible now and into the future. As a sub-project, it is also considering procedures for preserving physical format electronic materials.
For some months, the Library has been assessing online electronic materials and identifying those that have national significance. Some initial work has been done on experimenting with ways of cataloguing, organising and managing them so that they can be made available to potential users. Guidelines have been established for selection for preservation and titles are being selected in accordance with those guidelines. To date, around 120 titles have been selected for preservation and archiving. These titles have survived both the preliminary selection round, and a further in depth analysis against the selection criteria. The in-depth analysis phase has seen attributes like content, reliability, stability, form and format recorded.
There is still work to be done to establish conditions and procedures for the long-term access, management and preservation of electronic materials. This includes establishing cataloguing and bibliographic systems that take account of the needs of electronic materials and to formulating conditions for maintaining and managing an online electronic archive within a library environment. Possible data management systems for the electronic archive are currently being researched.
There are many issues to be finalised, involving both intellectual and physical access design. These include:
- negotiating the right to copy for preservation purposes
- consideration of fair compensation for use
- preventing unfair use and reproduction
- maintaining the integrity of the information contained in the documents
- capturing information in an appropriate timeframe
- maintaining an audit trail of information on archiving
- making access as easy and quick as possible, consistent with the factors above
- looking towards migration strategies as current technology ages.
However, considerable progress has already been made, and we are looking forward to continuing to cooperate with the owners of electronic information to make the Pandora archive a valuable resource for Australia.
Appendix - Some Examples of Electronic Publications
There are many examples that could be used to discuss the importance of Australian electronic resources to the law, the variability of their formats, and the difficulties posed for long term preservation and access. The following three have been chosen to demonstrate some of the ideas discussed above.
Australian Legal Research Library on CDROM (Butterworths)
This title (including a component title Halsbury's Laws of Australia) has been placed on voluntary deposit in the Library since 1995/6. The Library also purchases copies of the product for use in by readers, since it has been identified as a high use product.
The deposit copies are kept in the closed stack area of the Library and are clearly annotated as 'for use on stand alone PC only'. At this early stage of their life, the information contained in them is still readable, and the physical objects are in good condition. However, as technology changes, the information contained in these CDROMs and associated software will need to be migrated into an appropriate format. Already, DVD or digital video disc format is attracting considerable publicity.
E-Law, the Murdoch University Electronic Journal of Law - ISSN 1321-8247
This journal, covering general law and legal issues, commenced in 1993. At that stage it was a simple text based publication - in simple black typewriter- like characters on a grey background, available by ftp or email. At that time, advanced graphics-based browsers like Netscape Navigator and Microsoft Explorer were still some time away.
Since Volume 3 No. 1 (May 1996), it has been Internet based with a simple but effective presentation and access to all issues is possible from the title homepage. The earliest issues are still in ASCII text, but there are direct hypertext links to them - no more need for ftp or email.
As an indication of the importance of such a journal to its client group, its Netwatch column, by Anne Greenshields should be noted. This started in 1995, and lists with brief annotations information on the Internet related to law. In the latest issue (December 1996) this column runs to nearly 6 printed pages, ranging from Australian legal sites and developments to technology and the law.- a total of 39 online hypertext links to legal information.
E-Law is being indexed by AGIS, and has also been selected by one overseas site as part of a trial arching project of scholarly journals.
E-Law fits clearly within the Pandora guidelines, and has accepted our invitation to be archived by the National Library of Australia .
The Australasian Legal Information Institute (AustLII)
AustLII is operated by Faculties of Law at the University of Technology, Sydney (UTS) and the University of New South Wales (UNSW), with funding from a variety of government and other sources. As noted earlier (AustLII FAQ 7.6):
AustLII aims to put on the Net 'public legal information': primary legal materials (legislation and decisions of courts and tribunals); and secondary materials that are (or ought to be) in the public domain or able to be licensed for free.
AustLII is an example of a site that has changed its location. Originally at http://austlii.law.uts.edu.au/ , a redirect message says As AustLII grows and resources need to be distributed, we need the flexibility of our own namespace. This redirect message is still active over twelve months since the change occurred.
Although AustLII saw as its primary audience Australian academic legal researchers, use statistics indicate that AustLII's users span the whole community, including educational institutions (about 30%), the legal profession and business (25%), community organisations (15%), government (10%), and 20% from overseas.
At this stage, each update of AustLII overwrites pre-existing information. This means that it is provides current, but no archived or historic information.
The AustLII site is what in the terms of the Pandora project we have been calling a mega-site. This means that it is so large and all embracing that is a library in itself - it currently contains around 500 000 individual documents.
At this stage of the Pandora project, we are not considering the total archiving of any mega-sites (partly because we do not have sufficient resources in place to deal with them), although we may selectively archive part of the site. Although in time this decision might change, if some organisation does not take up the responsibility to save the whole site, the information it contains will be lost to future generations.
