Staff Papers
International Cooperation - We are all in the same boat so let's row together
Paper by Jan Lyall, Director, National Initiatives and Collaboration, National Library of Australia, presented at RLG 1997 Annual Membership Meeting - Chicago 5-6 May : Digital Futures - Ensuring Long-term Access to Digital Information
Introduction
Ensuring long term access to digital information is not a simple challenge. Indeed in preparing for this talk I was reminded of a story recounted by Klaus-Dieter Lehmann at a recent conference in Melbourne. Since the story was told to Klaus-Dieter by Don Waters, some people here may already have heard it, but I want to repeat it because it describes the feelings I sometimes have when trying to plan a course of action in this area. The story goes like this:
One morning, concerned that her son did not appear to have woken, a mother went to her son's room and found him still in bed. 'Are you all right?' she enquired. 'I'm OK' he replied, 'but I'm not going to school today!' Being a modern mother she decided to reason with him. 'Well then', she demanded 'give me three good reasons why you aren't.' 'OK' said her son.
'1. I don't like school,
2. The teachers don't like me and
3. I'm afraid of the kids'
'OK' said his mother, 'now I'll give you three reasons why you should go:
1. I'm your mother and I say school is important
2. You are 45 years old and
3. You are the principal of the school.'
Even though the task in front of us is sometimes overwhelming and the amount of available information so vast, we can't all 'stay home' and let others solve the problem. One reason some of us keep trying to find our way through this new maze is that despite the difficulties, the challenge is exciting. There is no doubt that the ground-breaking work of the CPA/RLG Digital Archiving Task Force has made the task a lot easier for all of us. Its report Preserving Digital Information has identified the full range of issues to be confronted in developing strategies for providing long-term access to digital information. We are all genuinely indebted to them.
Previous speakers today have looked at organisational, technological and intellectual issues associated with preserving access to digital information and my task is to tease out those aspects which require international cooperation. I doubt that anyone would now deny that the issue of digital preservation requires cooperation. The tricky bit is to work out what issues require international cooperation, which ones are best dealt with on a national basis and which can be solved within an institution. The fact that so many issues are already being tackled at an international level demonstrates that this has already been recognised and we are beginning to develop solutions to some of the problems.
One of the barriers hindering international cooperation is that the immediate pressures of our daily work usually take priority over a responsibility to respond to or make contact with someone on the other side of the world. It is often difficult enough making contact with someone in one's own organisation, city or country! Cooperation will only succeed if all partners benefit.
In this paper I will:
· attempt to identify areas in which international cooperation is desirable,
· propose a few projects where none yet exist, and
· put forward a model for determining partners and strategies.
A large part of my paper describes work that has been carried out in Australia. I think that much of it can be 'scaled up' to the international level but I leave that decision up to you.
Principles - http://www.nla.gov.au/npo/natco/princ.html
Throughout 1996 in Australia representatives from libraries, archives, publishing companies, museums, film and sound archives, governments, the research community and the information technology sector refined a set of principles, the first draft of which was examined in November 1995 at the National Library of Australia's National Preservation Office's 'Multimedia Preservation - Capturing the Rainbow Conference'. The Statement of Principles: Preservation of and Long-term Access to Australian Digital Objects, which was released early this year, recognises that different strategies will probably be needed for the various categories of digital objects, but assumes that all strategies will be underpinned by the same set of principles.
In preparing these principles it has been assumed that Australian digital objects form part of the national 'imprint' and as such must be preserved by Australia. This is in accord with the IFLA (International Federation of Library Associations and Institutions) policy which states that each country is responsible for the preservation of its own national imprint. The European Union has also recently acknowledged that individual EU-member states have the primary responsibility for the preservation of their own national cultural heritage. This emphasis on preserving national materials is common amongst national libraries and differs from the approaches of many large research libraries in North America and Europe, where the emphasis is likely to be on preserving materials in a particular academic discipline.
The seven principles are interdependent and of equal importance. For the sake of convenience and to provide a structure for examining areas in which we can cooperate, I will deal with each principle in turn, giving examples of how it can be seen to underpin particular projects or activities, and where possible, suggest international strategies building on that principle.
Principle 1: Cooperation
The cooperation of all with a role in the generation, use, management, distribution and preservation of digital objects is essential.
Continuing access to digital objects will be dependent on the cooperation of information creators, information system designers, manufacturers, publishers/distributors and information custodians and providers.
This may appear to be stating the obvious but there is a need to expand the extent of consultation and agreements. The explanation beneath this principle identifies some of those with whom we will need to cooperate.
Many libraries around the world and indeed many Australian libraries are attempting to ensure that the digital information being created today will remain accessible in the future. The National Library of Australia (NLA) has taken the initiative on a number of fronts and I will describe the most important of these.
NLA's initiatives in cooperation
· National Library's Strategy paper - http://www.nla.gov.au/policy/paep.html
· PADI - http://www.nla.gov.au/padi/
National Library's Strategy paper
Late in 1996 the NLA released a position paper National strategy for provision of access to Australian electronic publications. Using conventional mail and electronic distribution the paper was made available to a large number of libraries and other relevant bodies. The paper recommends that libraries, archives, producers and creators cooperate to ensure long-term access to electronic publications. While recognising that deposit libraries will continue to play a key role, it recommends that others, particularly university libraries, should accept responsibility for providing long term access to their own publications. The paper also makes recommendations on the need for changes to legal deposit legislation, the role of cooperative collection development, access, copyright and fair dealing issues and issues associated with organising electronic publications.
The whole process was very beneficial to the NLA. The conceptual thinking assisted us in clarifying our ideas and has enabled us to move on to the next stage where we are actually doing things rather than just talking about them.
The release of this paper has clarified the full spectrum of issues and given the Australian library community the opportunity to discuss the issues and provide comments back to the NLA. Comments have been generally supportive and in particular there was strong support for collaboration. There was also a view that NLA was the appropriate organisation to act as leader
It has raised the level of awareness and has positioned the NLA to take further action but it is fair to say that it has not set the country on fire! It will be up to the NLA to ensure that action follows.
Preserving Access to Digital Information (PADI) Working Group
In today's terms where anything older than a few years is considered outdated, the Preserving Access to Digital Information (PADI) Working Group, which I chair, has had a long history. It was set up in 1993 as a cross sectoral group, involving representatives from the library and archival communities. Its brief was to develop guidelines for the management of material in electronic format. The group struggled for a number of years, gradually expanding its membership to include representatives from the museum, information technology and research communities, but never coming to grips with its task - probably because the task was particularly difficult and the group never had adequate resources.
The group received a great 'shot in the arm' in December 1995 when it received funding from the Australian Vice-Chancellors' Committee. A decision was made to employ a consultant to work with the working group to develop the guidelines, which at that stage were envisioned as a printed document. It will probably come as no surprise to you that the consultant was not the answer to our dreams! In trying to explain the full extent of the problem to the consultant we learnt a lot. The very broad nature of the group exposed all of us to situations which we, as individuals in our own organisations, had not had to deal with. In the process we clarified our thinking and slowly edged our way towards a new approach to our task.
Late in 1996 the group decided to abandon the idea of producing printed guidelines and to develop an evolving electronic document which would contain original material as well as links to relevant sites in Australia and elsewhere. We parted ways with the consultant and employed Leanne Brandis, a scientist who had worked in the library preservation field for a number of years. Considerable time was spent designing the Web-site to provide a logical yet simple structure which would be easy to navigate and be relevant to everyone from concerned creators to preservation specialists. The aim of the Web-site is to provide guidance for preserving access to digital information.
We launched the Electronic Proof of Concept (EPOC) site early this year and shortly thereafter ran out of funds. Most PADI members contributed a small amount to tide us over until we found out the results of our next grant application. The results of this bid have been delayed and the NLA has been paying for Leanne's services for the past three months. As the PADI Web-site has developed it has become more relevant to the international community. In the process it has changed its name - PADI originally stood for Preserving Australia's Digital Information.
As well as developing and maintaining the Web-site PADI aims to:
· establish a discussion list
· facilitate the development of national strategies
· influence the preservation aspects of national digital information initiatives
· develop preservation and access guidelines and strategies
· identify and promote relevant activities
I believe that if PADI is achieving its goals there is little point in others setting up similar Web-sites. There is much to be done to improve the site but the development of this site provides an ideal opportunity to work collaboratively at an international level. There are few boundaries in this field and where they exist it is a simple matter to add country specific information.
National cooperative project: http://www.nla.gov.au/ferg/
Many digitisation projects are taking place in Australia but most are intended primarily to improve current access. However there is one cooperative project which has long-term access as one of its major goals. It is the Australian Cooperative Digitisation Project which is a collaborative project between the University of Sydney Library, the State Library of New South Wales, the National Library of Australia and Monash University Library. It involves the digital conversion of Australian journals, newspapers and fiction of the period 1840-45 - a period significant in the emergence of an Australian colonial identity. A hybrid approach involving the production of both preservation microfilm and a digital product has been adopted. This approach is similar to that being employed in projects in other countries.
In recognition of this fact international advice has been sought throughout the course of the project. Since several projects in different parts of the world are proceeding along similar lines it is likely that mutual benefit would be derived by sharing information on technical and management details associated with such projects.
The PADI site is a possible mechanism to share this type of information. It already has a section called 'case studies' which contains brief information on and links to a number of projects.
International Cooperative Projects
RLG has initiated several international cooperative digitisation projects and while the primary emphasis of Studies in Scarlet and the Migration programs may not be preservation, preservation issues are being taken into account. The RLG Arches project aimed at improving methods of archiving digital information will undoubtedly support these large international projects.
The UNESCO Memory of the World Programme is an ambitious program whose goal is to ensure the preservation of valuable archive holdings and library collections all over the world and to provide access to them by exploiting modern technologies. Information on the programme is at: http://www.unesco.org/webworld/mdm/en/index_mdm.html
Principle 2: Role of creator
Creators of digital objects have the initial and in some cases a continuing role in preserving access to them.
Unless creators take initial steps to facilitate ongoing access to their material, access may be lost or the ability of another body to take responsibility for preserving access may be severely inhibited.
This principle has been rather contentious. Several respondents to the draft version of the Principles commented that most creators would be unwilling or ill-equipped to assume this responsibility. From the perspective of a national library, we are concerned that unless creators are aware of the consequences of their actions, we will be unable to provide access to the digital objects they create. The NLA does not expect to be responsible for the preservation of all Australian digital information. Its Position Paper recommends that the responsibility for archiving some categories of digital information should remain with the creator. This is seen to be relevant for much material created by universities. To date no university has accepted this challenge!
In the meantime, an attempt to resolve many of the issues associated with ensuring long-term access to digital information and to demonstrate that such things are possible, the NLA has embarked upon a project to archive and provide access to significant Australian online materials. This project - the Pandora project - Preserving and Accessing Networked DOcumentary Resources in Australia - is a project to investigate strategies for the storage, preservation and access to digital information by establishing a digital archive at the NLA. This work is a direct consequence of the Library's responsibility to collect and preserve electronic Australian publications of lasting cultural value.
Pandora project - http://www.nla.gov.au/policy/plan/pandora.html
The PANDORA team is selecting current Australian digital publications which it considers to fall within our collecting policy. Both online and physical format types (ie CD-ROMs and floppies) are covered. Since electronic publications are not included in the present Legal Deposit Act we are making contact with the publishers, seeking approval to archive their materials, exploring ways of having them include metadata and negotiating conditions of access. Most publishers have willingly given approval for their material to be archived. No major commercial publisher has yet been approached, but we hope to also gain their support. In the case of online material, rather than ask for a copy to be deposited, the Library seeks permission to capture the publication, as it exists on the Internet, into the Library's own digital archive.
The PANDORA project is providing a practical demonstration of how the Principles are likely to work in practice. As such, it is relevant to most ,if not all Principles. I mention it here first because it obviously is exploring the role of the creator, but it is obviously relevant also to Principle 1 - Cooperation and I will be mentioning it later in relation to some of the other Principles.
The methods adopted in this study are likely to be of interest to others who are making initial approaches to publishers. In particular the study to be carried out in the UK to investigate the attitudes of originators and rights' owners to digital preservation responsibilities may be able to use some of the information we have collected.
Principle 3: Distributed responsibility
The location, selection, identification/cataloguing and retention of digital objects will be best achieved through the coordinated distribution of responsibilities.
A national network of archives, collecting organisations and other information providers operating through formal agreements in which responsibilities are well-defined will best serve to preserve access to significant digital objects.
Although there is general agreement to this Principle, it has not yet been implemented in Australia. Assuming that the legal deposit legislation will be amended to include material in digital format, the existing system of deposit libraries should ensure that at least one copy of Australian material will be preserved. In addition, based on the findings of a recent survey some responsibilities are likely to be taken up by the university sector. The survey carried out in mid-1996 indicated that about 50% of the responding university libraries were prepared to accept a role in the long-term preservation of digital information. To date none have yet formally agreed to accept any responsibility.
In Australia the issue of whether it is necessary to archive more than one copy of an electronic publication has not yet been addressed, but it is expected that this issue will be discussed on the new closed electronic discussion list of the National and State libraries.
In the print environment there was rarely any need to look beyond national boundaries to develop shared responsibilities for preservation. In today's networked environment where megasites are created using material from various countries we will be forced to develop procedures to share responsibilities. RLG is ideally situated to lead such initiatives.
Conspectus - http://www.nla.gov.au/dnc/conspect/modpres.html
The recent development of a system to record preservation priority codes into the Australian Conspectus has the potential to facilitate cooperation and shared responsibility. It has been designed to be relevant to both conventional and digital materials. It will enable a national picture of preservation activities to be developed and will help to identify areas of responsibility. It will also assist in locating material for which preservation responsibility has not been taken. Despite all of these advantages it has not yet been adopted by any library! We would welcome any comments on the relevance of this approach.
Principle 4: Appraisal
Access to digital objects should be preserved only for as long as they are judged to have continuing value and significance.
It is neither practical nor desirable to maintain access to all digital objects indefinitely. In applying professional judgement to appraising items, consideration should be given to balancing costs and benefits, including the likely needs and interests of future generations.
This Principle is saying we can't keep everything forever. As Paul Conway has said 'Preservation adds value through selection'. Everyone know this is true but all libraries have difficulty in deciding what is not to be kept. The National Library of Australia has developed guidelines for selecting online Australian publications for preservation. This difficult task was carried out by SCOAP (Selection Committee on Online Australian Publications).
SCOAP - http://www.nla.gov.au/1/scoap/scoapgui.html
The SCOAP committee was instructed to not be constrained by technical or resource issues. It took the existing Collection Development Policy for print Australiana as a starting point for its deliberations. The exercise was very resource intensive with committee members surfing the Net to become familiar with what was there. It was decided that the format of an item should make no difference to whether or not it was preserved - content of an item was the paramount issue. In order to make decisions the fundamental issue which had to be resolved was deciding what constituted an Australian publication.
This preliminary work has enabled the PANDORA project to be carried out. To date, 150 titles have been selected for archiving. Attributes such as content, reliability, stability, form and format have been recorded. It has been decided not to attempt to preserve megasites. In addition, procedures and selection guidelines for the preservation of physical format electronic materials are also being developed.
The issue of selection will need to be discussed at several levels. National libraries will need to examine their roles as repositories of their national 'imprint'. Institutions involved in cooperative projects will need to develop criteria for selecting material to be preserved and individual institutions should make decisions based on local needs. Cooperation will be required at the international level for partners in projects and all concerned would benefit by having access to information of how others were dealing with the problem. This type of information could be either placed on or links provided from the PADI-Web-site section 'What is to be preserved'.
Principle 5: Rights
Rights of creators, owners, providers, users and subjects of digital objects must be balanced and protected.
The preservation of significant digital objects should not be jeopardised by censorship of form or content. Rights of access and the right to receive benefits arising from the digital object need to be carefully balanced and protected.
There are too many issues in the area of rights to discuss in this paper. Action is required at national and international levels on issues such as copyright, legal deposit, moral rights, fair dealing and provision of access. In Australia the Copyright Act is under review and the NLA has made submissions to this enquiry. The joint response by the NLA and the National Film and Sound Archive (http://www.nla.gov.au/policy/clrcld.html) in relation to legal deposit clearly sets out the ideal situation for legal deposit institutions. Activity within the World Intellectual Property Organisation (WIPO) is relevant to all nations.
Although many of these issues have a direct impact on preservation strategies they have much wider ramifications. In particular they affect the large and powerful publishing industry. From the experiences at the NLA there is nothing quite like confronting these issues head-on. The PANDORA project has enabled us to resolve issues such as voluntary deposit, copying for preservation, fair dealing and provision of access in a manner satisfactory to both parties, although it must be said that the large commercial publishers have not been approached yet.
Principle 6: Strategies
The adoption of best practices and standards is essential.
Strategies to preserve access to digital objects must consider the integrity and functionality of these digital objects. Standard metadata elements should be used to ensure access to the information. Continuing research and dissemination of findings is essential. Strategies to preserve access to digital objects will need to be periodically reviewed and modified to reflect current best practice.
Owing to the speed with which technological changes are occurring it is impossible to predict what the most effective methods for preserving access to digital information will be. Our work with PANDORA is an attempt to develop best practice for the present.
One critical issue which is receiving attention is that of permanent naming. Action is taking place on a number of fronts.
Permanent naming
· URN - Universal Resource Name - IETF (Internet Engineering Task Force)
· PURL -Persistent URL - OCLC
· Proposed Working Group of CDNL (Conference of Directors of National Libraries)
· Digital Object Identifier - DOI - Association of American Publishers
The National Library of Australia is establishing a PURL resolver service. It also plans to take the initiative in establishing an international working group to examine the issue of permanent naming. It will put a proposal to CDNL (Conference of Directors of National Libraries) to establish a small working group of interested national libraries.
Standards
· Z39.50 - data search and retrieval
· Metadata - Dublin Core
· World Wide Web Corporation - W3C
· Others too numerous to mention (see PADI)
A wide variety of standards of all types will be required. Best practices will only be established if enough real projects are commenced and the lessons learned are communicated to others. Projects such as Studies in Scarlet and PANDORA have solved some problems and identified others. The identification of problems has resulted in research being commenced in a number of areas.
As more multimedia material is available on the Internet it will become increasingly difficult to search for and retrieve the information we are seeking. Developments in automated indexing procedures will assist but research is required to investigate methods of searching across global systems for multimedia materials.
Principle 7: Role of government
Appropriate Commonwealth and State government regulatory, legislative and policy regimes are essential.
The continuing and equitable access to digital objects must be supported by appropriate regimes, including legal deposit legislation.
A number of areas which will require government intervention have already been mentioned but the critical issue of the costs associated with developing digital information systems has only received scant attention by most governments. The possibility of sponsorship, especially for digitisation projects involving 'national treasures', has been explored by several institutions. This approach is one favoured in the Memory of the World Programme. With sponsorship there is a risk that some less developed nations may relinquish the rights to their images.
MAKING PROGRESS
The Principles are useful in distilling out the essential elements but they are not adequate for developing the detailed strategies required to deal with the new environment. Cooperation is vital but it is not always easy to identify likely partners and to isolate those issues which are best worked on with particular partners.
One method used recently in the NLA to identify possible partners has been to develop a rather complex matrix. The first step was to divide the life cycle of digital objects into four broad stages:
Life Cycle of digital objects
· Getting
· Organising
· Keeping
· Accessing
These distinctions have been made while recognising that in reality there are few distinct boundaries between these areas but the division has proved useful for this exercise. These four basic elements were further divided into separate components. Issues associated with each component were then identified. For example in the case of the getting and organising stages the next steps are illustrated on the next two slides .
Getting
· Identifying Active searching
Notification of existence
· Creating Authoring
Digital publishing
· Selecting Collecting agreements
Selection criteria
· Acquiring Legislation
Physical objects
Licensing
Networked information
Costs
Organising
· Intellectual and physical control Standards eg cataloguing
Metadata
Changing URL
· National register Standards
Use of common system
Rather than describe the components and issues for keeping and accessing in my talk, they have been included in the printed handouts you received in your kits.
Having listed the issues it is a relatively simple matter to identify potential partners and to propose possible strategies for dealing with the issues. I will not discuss this in any detail but offer it as a possible way of identifying partners and strategies for international cooperation.
Other issues
There are several important issues I have not discussed in this talk. Some I avoided because I do not know enough about them, some I avoided because they are too complicated and some I avoided because there was insufficient time to do them justice.
However, before concluding I will briefly mention some issues which I believe will influence the development of new methods for operating across institutional, national and international boundaries.
The first is the uncertainty of predicting the effect technological change will have on the structure of our organisations.
The second is the need for new training programs. Technological and organisational change will create massive training needs.
The third is the difficulty in predicting how the economy of the Web will affect the delivery of digital information. We can confidently predict that the Web will evolve to permit electronic payments. In fact it has been predicted that by the year 2000 revenues from electronic payments will account for two thirds of all non-cash payments in the US. But will information as we know it always attract buyers? In the April issue of D-Lib magazine Bill Arms asks 'Is information a commodity?' On the Web creators give away information. Making that information easy to find is becoming a major problem. Already the amount of information available is enormous. Perhaps we will be faced with a situation where we have to make our information attractive enough to be found and used, rather than putting a price on it. So much of our current thinking is based on the world as it was in pre-digital times.
The last issue I wish to mention is the role of developing countries. We are operating in a largely English speaking, technologically advanced society but countries such as India, China and Malaysia are likely to be major players in the coming decade. How this will affect us is not known but I suggest that we should be including them in our discussions.
Conclusion
In looking at the current environment we have all concluded that progress can only be made if we cooperate at both national and international levels. The problem is that we have not yet found the best way to work together. Will we be able to develop effective means of cooperation if our communication is mainly electronic and at a distance?
Most of us work in organisations where we depend on personal contact to establish work teams. We rely on being able to interpret the behaviour of others. Things such as changes in tone of voice, body language, and facial expression influence how we interact with others. Meetings provide us with an opportunity to achieve more as a group than we can as individuals. All of this is very different if we establish a work team where members have not met and are communicating by e-mail. International travel is expensive. It has an adverse effect on an individual's performance and is time consuming. However, electronic communication will have to improve dramatically before we can dispense with meetings such as this.
At a very practical level, RLG seems to be ideally placed to establish new methods for international collaboration. It could emphasise this role by altering its management structure and celebrate the change in direction by adopting a name. Collaborative International ResearCh Libraries Enterprise - CIRCLE ?
