Web archiving in a fast moving world

The tricky task of capturing websites while the news is hot

When the winds of political changes flow through Canberra it becomes a busy time for the web archiving team. We have blogged about the timeliness of web archiving before, but that was for one site, the events of last week impacted upon many sites in a short period of time. Once the news broke of a Caucus ballot for the leadership the online news services started producing online content at a prodigious rate as they covered the rapidly evolving story.

Most often the changes of government occur at set times and we have a date to work to, namely election day, but at times these changes occur unexpectedly like they have in the last week when the the Prime Minister and a number of Ministers have been replaced or resigned in a matter of hours or days.  These changes will all be reflected online as websites are updated, Ministerial positions are reviewed and news sites report on these happenings.

Prior to the digital age the National Library of Australia's main source of information on events such as these would have been the print newspapers that arrived the following morning. But between the events and the headlines of the next days morning paper the online news services have displayed and removed many news articles and headlines as they followed the evolving story. In the online world the Library collects this material through its web archiving activities.

So how do we capture all the changes that occur through an event like that of the evening of the 26th June 2013? Our first priority is to capture online news content, focusing on online news sites that we have obtained permission previously like the Sydney Morning Herald and the ABC. We cannot capture every change and so must choose points in time that will be most relevant, each twist of the political story, the rumours (see below, capture at 1.28pm), the announcement of the caucus ballot (capture at 4.56pm), the interval as the nation waited for an announcement of the winner (capture at 7.17pm) and the aftermath (capture at 10.05pm). All of these generated new headlines and articles as the events unfolded. Luckily our web archiving tools are browser based and this means we are able to log in from home and set manual gathers at times day or night when we are not at work to ensure we have these timely archived copies.

Sydney Morning Herald, 1:28pm Wednesday 2013

Sydney Morning Herald, 4.56pm Wednesday 2013

Sydney Morning Herald, 10.05pm Wednesday 2013

The ABC has had ongoing coverage of the Labor leadership challenges over the course of the last three years. As each challenge took shape the ABC created a new website that linked to all the content and articles the ABC produced. These sites were refined over the years with the earliest one, in June 2010, using the standard ABC news banner, but for later challenges these were replaced with tailor made banners. During the 2012 challenge they used a banner entitled , "Showdown for PM" for the main site of the ABC coverage.

ABC News, 7.08am 22nd February 2012

In March 2013 ballot the ABC created a new banner, "Labor in Turmoil". This was reused just three months later for the final challenge of June 26.

ABC News, 1.53pm 21st March 2013

ABC News, 6.52am 27th June 2013

This culminates in the ABC banner, “Rudd returns” on the day following the successful final challenge.

ABC News, 12.50pm 27th June 2013

In addition to the main news sites that we archive major commentary sites (for those we have permission). These include the ABC’s The Drum, The National Times, Crikey etc. By making a timely archival copy of these sites at the time of the events we ensure future researchers have the contemporary resources they require to dissect, evaluate and write the history of our times. After the event we find our attention turns to the changes that have occurred in government and how this will be reflected in the governments online presence. In the rapidly evolving political world government websites can change quickly. A final archive copy was made of Julia Gillard’s Prime Ministerial website (pm.gov.au) at 6:41pm (a precautionary gather taken before the result was known and it would have been discarded had the result been different) on the evening of the challenge, June 26. As Julia Gillard is now leaving politics at the next election this becomes the final copy we make of this site before it is shutdown. Prime Minister of Australia - Julia Gillard The next day this site was removed and changed to Prime Minister Kevin Rudd after he was sworn in by the Governor-General. We were able to get a copy of this new site at 11:48am. This is a continuation of the previous title we had created for Kevin Rudd PM starting in 2007. By responding to events as they happen we can ensure we have as accurate account of the websites as they appeared and disappeared. Prime Minister of Australia - Kevin Rudd The change in leadership has also resulted in numerous Ministers resigning from the cabinet and new ministers being appointed. Ministerial sites are complex beasts for web archivists. Portfolios can be created and can disappear, they can be split and merged and the Minister responsible can come and go from parliament or be promoted or demoted. I won’t go too much into Ministerial sites, that is for a future post, but the new Minister's sites will be reviewed and changed as they are announced and their sites updated. We have begun to curate a Ministerial collection that will help bring some order to the over one hundred Ministerial sites we have archived since we began our work in 1996.

Add new comment