Staff Papers

DIGITAL DEMONS! The Challenges Facing Online Publishers

By Roxanne Missingham published in Informaa Quarterly Magazine (Record Management Association of Australia) November vol 21 no 4

There are demons in the works of the Web, and earlier this year a forum was held in Canberra to discuss growing problems associated with publishing government documents and reports online. It generated great interest, and, as this report demonstrates, highlighted significant problems now facing anyone publishing online.

The Digital Amnesia Forum was organised to discuss and identify issues relating to digital government publications and was requested by the Peak Bodies Forum of Australasia’s peak library organisations, which had identified significant issues facing the Australian library sector, and developed a national plan of action.

Over 150 participants from Australian states and territories and New Zealand attended this seminar, held by the Australian Library and Information Association (ALIA) and sponsored by the National Library of Australia and the National Archives of Australia, on Thursday 21st April 2005.

Patrick Callioni, Division Manager, AGIMO, Department of Finance and Administration, set the scene with a keynote speech about the prolific nature of current Australian government publishing and the nature of the federated approach where all agencies have responsibility for their own publishing.

The overall policy, he says, is to make all government publications and services available online. There are many legal obligations contained in the National Library Act, Archives Act, Copyright Act, Evidence Act and Freedom of Information Act. The closure of the government bookshops, due to lack of use, has led to a rethink in the roles and issues in providing access to government publications.

To Callioni’s mind, critical issues include understanding what a publication is and taking actions to overcome any possibility of digital amnesia, or loss of electronic publications. He attributes the cause of digital amnesia to the decrease in hard copy, increased electronic communication, such as email, and inadequate preservation of it, combined with poor recordkeeping and data management practices.

Devolution of responsibility for publications makes it harder to track and manage publications, he says. There is also a failure to appreciate new forms of publication. He identified two problems – discovery and retention. Discovery issues, he says, are critical for the success of www.australia.gov.au.

While metadata is the basis for the underlying structure for discovery, Callioni proposed that a semantic web would emerge in the longer term, based on alternative forms of data. Permanence of resources is a more difficult issue because of the lack of guidance and controls.

Callioni pointed out that there are approximately 1150 government web sites, many with different search engines, interfaces and styles. There are many questions that need to be asked. Are we using the best technology? The right technology? And are resource discovery mechanisms effective?

A strategic review of metadata is being undertaken by AGIMO, the National Archives of Australia and relevant stakeholders. AGIMO will work with the National Library of Australia to improve the coverage of the publications website and develop strategies to ensure that agencies register all their publications with the National Library.

Callioni forecast that technical storage issues, including technical obsolescence, may never be completely resolved and that policy and guidance must be established as the cornerstone for changing attitudes towards electronic publications and their retention. Discovery mechanisms are developing rapidly and will help address issues of communication with the public. Permanence, he said, is the new challenge for immediate attention.

Jack Waterford, Editor-in-Chief of the Canberra Times, commented on the difficulties faced by the media in finding government information and publications. He suggested that web publishers are not, by and large, readers and therefore run the risk of creating websites that are not truly user friendly.

He recommended that media savvy users be taken into account, as research into Canberra Times readers had found that many read several papers, listened to news on the radio and television and also used the Internet.

Waterford described the frustration of trying to find older material, having to use resources which were published on the Internet in individual chapters, and publications which are surrounded by marketing material. He expressed concern about a lack of managing for permanence exemplified by gaps in ministerial records.

Ian Oi, Special Counsel with law firm Blake Dawson Waldron, noted that digital amnesia "rips holes in the commons" through link rot, where links break and searchers can no longer locate resources. He described the importance of Creative Commons, such as the AEShare initiative to ensure long-term free access to resources through alternative copyright arrangements.

He says that an information commons approach enables storage of resources and depends upon effective tools, such as legal deposit. The lack of legal deposit for digital publications puts long term access as risk. Equally important are adequate means for finding resources and preserving them in context, particularly older content, such as past versions.

Oi noted that increasingly government is covered by a complex range of legislation, and e-government activities may create new rights and obligations, adding more complexity. And there is potential conflict between the imperative to tackle legal amnesia and legislative requirements.

Kerrie Burgess, Research Coordinator with the State Library of New South Wales, described research into the role of public libraries in providing support for e-government.

Helen Hopkins, Executive Director of the Consumers’ Health Forum of Australia, spoke of how her Forum members are concerned about information overload. The Internet has provided increased access to information, however issues such as printing and reading large documents, actually finding information and having appropriate information are critical.

As an example of the problems faced by online information seekers, Hopkins told of how the Department of Health's website did not provide basic facts about generic medicines, although Healthinsite provided some information links.

Hopkins was also concerned that her Forum contributed to reports which either are not published or disappear. For example, a trial of Mediconnect health records had not (by the time of the seminar) resulted in a published record of the extensive research, which could mean that the next stage is not informed by this research.

A focus group held in the Western Sydney suburb of Campbelltown had found the public wants Web access, but find too much information. And it is hard to judge what is quality information. In addition, websites, Hopkins points out, do not always cater to the needs of inexperienced users.

‘A new set of questions needs to be asked about online publications including whether appropriate technology and delivery strategies are being used’

Senator John Watson, Chair of the Senate Publications Committee and member of the Standing Committee on Publications, noted that his Committee is interested in the availability of both print and online material, including issues such as distribution, preservation and metadata.

He also spoke about the challenge for all agencies including dealing with versions of reports, cross linkages and making more use of metadata for resource discovery.

Senator Watson expressed concern over the lack of legal deposit of digital publications which preclude easy collection and preservation in the National Library’s digital archive, PANDORA.

Toss Gascoigne, Executive Director of the Council for the Humanities, Arts and Social Sciences (CHASS), described research which indicates broad concerns about the disappearance of Web resources and the complexity of providing access to current and older materials. (See Toss Gascoine’s article ‘Has Web Publishing Become a Can of Worms,’ earlier in this issue of IQ.)

Steve Matheson, Head of the Information Services Branch with the Australian Bureau of Statistics (ABS), said that the ABS delivers its service extensively online. The complex ABS website, with approximately 195,000 web pages and over 110,000 downloadable files, had over 50 millions pageviews in 2003/04.

ABS print publications will continue to decline - over the next two years by approximately 15%. He described the ABS access model as a means of making sure that the website reaches all types of users. He was also one of several speakers who raised the question of what constitutes a publication for the future.

Matheson also stressed the vulnerability of data, which ABS addresses through retention policies. He also said the ABS has an outreach program providing access through public libraries which it supports with training and access to publications.

Adrian Cunningham, Director, Standards and Initiatives with the National Archives of Australia, discussed the NAA’s strategic approach to digital recordkeeping in government, including the Archives’ digital preservation project.

He described the role of Australian Government Locator Service (AGLS) metadata in supporting online resource discovery, noting that the semantic web initiative also relied on metadata supplied by resource creators or owners.

Cunningham said the Archives’ policy and guidelines for archiving web resources states that web archiving strategies can use a mixture of object-oriented or event-driven approaches to capturing web records. Appraisal disposition of web resources within the Archives’ records disposal authority framework must be undertaken by agencies. Preservation, he said, is vitally important for a strategy to ensure long-term access.

Pam Gatenby, Assistant Director General, Collection Management with the National Library of Australia, summarised the key issues of access and permanence. She noted that a survey of resources with urls in the National Library’s catalogue found that approximately 14% had "link rot" and no longer worked.

She noted that identifying, describing and storing resources required considerable effort and that while the Library’s PANDORA archive contained many government publications, it required more resources to expand its coverage.

Gatenby noted that the Library was undertaking a full snapshot of the Australian web in 2005. However, these resources would not be available to users unless the Library is able to negotiate blanket permission to allow users to access all government websites. She noted added the lack of legal deposit for online publications including government web sites meant considerable effort was required to obtain permission for access.

Keitha Booth, of the e-government unit, State Services Commission of New Zealand, reported that the NZ Government’s initial e-government policy was that the Internet would be the dominant means of enabling ready access to government.

By 2007, she said, networks and Internet technologies would be integral to the delivery of NZ Government information, services and processes. A progress report in late 2004 concluded that all agency websites provide access to government information.

The e-government unit manages functional (FONZ) and subject (SONZ) thesauri and maintains a central repository for the .govt.nz portal metadata. At least one keyword from each thesaurus must be included in each record and there is a quality assurance process for approving service and document records on the portal.

Booth noted that New Zealand has mandatory requirements for website accessibility and has funded a national digital heritage archive managed by the National Library of New Zealand.

‘He attributes the cause of digital amnesia to… poor recordkeeping and data management practices’

Key Issues That Emerged

The Australian government is now primarily online as part of the Government Online strategy. Publishing is predominantly online, and the strategic issues now emerging for consideration reflect a maturing in assessing online delivery.

A new set of questions needs to be asked about online services and publications including whether appropriate technology and delivery strategies are being used.

The aspiration of placing all services and publications online is near, however whether the delivery method is appropriate has yet to be measured. Complex issues exist in relation to audience needs - some, such as health care consumers, have sophisticated needs for accurate detailed information.

Analyses of types of users, such as that done by ABS, give more insights, and new or novice users require greater assistance. Service models for resources and e-government services are still in development and need to evolve to meet changing understanding of consumer need and profile.

Legal issues, including the use of creative commons concepts should be explored to ensure access to government resources.

Libraries, particularly public libraries, can enhance access and utilisation of e-government. Adequate access, however, through public terminals and support through library staff assistance, requires greater funding.

Future resource discovery tools are likely to be influenced through the semantic web. These tools will rely on metadata which need to be intelligently and accurately applied.

Guidelines are required to ensure that government publications are reported to Australian Government Information Management Office (AGIMO) and the National Library Australia to enable information about these publications to be made available to the public in an acceptable and useful format.

The Web publishing process needs to incorporate the supply of information for resource discovery to support access and utilise cross linkages.

Permanent access needs to be addressed as an issue urgently, before more information and publications are lost – AGIMO, the National Archives and National Library all have a contribution to make to solving this problem.

Ongoing management of resources is required to prevent link rot, and to ensure that as organisations change their resources/publications remain accessible.

And, finally, the National Library Australia needs to have the right to collect, ensure access to and preserve electronic publications of all Australian government agencies.

The Author

 Roxanne Missingham is the Canberra-based Director of the Australian Libraries and Information Association, or ALIA. The Digital Amnesia Forum was conducted to further ALIA’s objective of ensuring free flow of information for Australians.

Back to the top