The National Library of Australia was one of the first national collecting institutions to take up the task of collecting websites and web documents for long term preservation and access (commonly referred to as ‘web archiving’). The NLA began selectively collecting web content for the PANDORA Archive in late 1996. Over the next decade other major Australian collecting organisations, including most Australian state libraries, became participants in building the web collection by taking on a share of curatorial responsibilities. In 2004 this pioneering work was recognised by UNESCO and the PANDORA Archive was listed on the Australian Memory of the World Register.
The PANDORA website as it appeared in 2004 when the Archive was listed on the UNESCO Australian Memory of the World Register
The PANDORA Archive collection was made openly available to the public almost from the outset, and after only a handful of web publications had been collected, in 1997 (through the catalogue of the time using the Z39.50 client-server protocol to link to the archived content). The NLA has maintained its strong emphasis on providing access to the archival content - surely the whole point of the archiving exercise - through full text search indexing and browse functions as well as controlled description for the library catalogue and national bibliographic database. To do both catalogue and full text searching of the web archive go to the NLA's single point discovery service Trove. While PANDORA is a selective web archive and therefore not a comprehensive collection of Australian web material, being one of the world’s earliest web archiving initiatives it provides a chronologically deep collection stretching back to the earliest days of the World Wide Web as an open public medium. Being established and operationally managed from within NLA’s Collection Management Division the Archive has been built with a collection development focus. Attention has been strongly set upon processes such as selection, obtaining necessary permission for access, timely acquisition (harvesting), quality assurance, description (cataloguing) and providing search based access.
The content of the PANDORA Archive will be a primary subject of this blog for the simple fact that its content is accessible and available for all to consult. However the NLA has for some time, since 2005 in fact, also been undertaking large scale web harvests, working with the Internet Archive to complete annual snapshot crawls of the Australian (.au) web domain. More recently, large seed-list based crawls of Australian federal government websites have been undertaken, in 2011 and 2012, with plans for the ongoing collection of this material. These government collections will be the first of the large scale crawls to be made openly accessible in the near future. One of the major challenges and objectives for a substantial and growing archival collection of web content is to improve the discovery mechanisms and to consequently enhance and extend use of the archive. While this blog does not of itself achieve this, it represents a recognition of this aim and is also an initiative to provide the experienced web archivists at the NLA and its partner organisations to explore and reveal aspects of the archival collection. The blog is intended for a wide audience and will be written with that in mind. We hope it will be of in interest of others involved in web archiving activities, but also to the Australian public generally and to other researchers. The blog will include short posts highlighting interesting or topical content discoveries, archiving case studies, and longer post exploring collections, content themes, issues and challenges. You can contact the NLA web archiving team at firstname.lastname@example.org