Impact of technology on archival programs: the National Library of Australia

Author: 
Marie-Louise Ayres
Publication date: 
Thursday, 19 December, 2013
Abstract: 

Dr Ayres presented this paper at the 2013 Australian Society of Archivists conference, in response to a request to speak about the impact of technology on archival programs at the National Library of Australia. This paper begins with a social media interaction between a member of the public and a 19th century piece of sheet music, digitised by the National Library and available via multiple digital channels, and traces the complex web of human, policy, investment and technology decisions and actions which made this interaction possible. The paper argues that archival organisations must embrace the cultural, policy and investment changes required to make their collections discoverable through multiple digital channels, moving on from a past in which archivists and librarians could expect users to come to either individual physical reading rooms or organisational websites to meet their research needs.

On Saturday 28 September 2013, AFL Grand Final, the Hawks beat the Dockers. Most of us could experience the game – before, during and after – through multiple digital devices, and in the Twitterverse the Grand Final even had its own hashtag.

Now I want to take you back more than a century to around 1879. The football polka, by W. Geddes Taylor was published by J.H. Sherring and Co, an Adelaide based printer responsible for a small number of lithographs, some poetry, programs, and even a lecture on women’s suffrage, delivered in Adelaide in 1895[1].

Viewing this piece of sheet music online, there are a number of things we can surmise from the artefact itself. It is dedicated to the Victorian and South Australian Football Associations. It is decorated with a rather charming engraving showing burly moustachio’d men wearing rather tight fitting gear. We appear to have some provenance, perhaps ownership by a Mr, Mrs or Miss Newbold of Ar-something. House, Florence Street Pt Pirie, and perhaps even an earlier owner at Gertrude Street – just one street back from Florence Street. We can also see the stamp of E. Quintrell, watchmaker, jeweller and stationer at the bottom of the front cover, possibly the E. Quintrell who opened a jeweller’s shop in 1913, and died 1929, described as a talented musician[2] or perhaps the E. Quintrell whose Kadina jeweller’s shop was daringly burgled in 1906[3], or perhaps even the earlier (or possibly the same) E. Quintrell who advertised his jewellery stock in the Wallaroo Times on 8 October 1887[4].

There are a few doodles, and a National Library mark showing it came into the collection in July 1999. Our Integrated Library Management system reveals that the Library purchased it (the rarest item in a job lot) from Pioneer Books, a South Australian rare books dealer from whom we still purchase. Alas, we know nothing about where they got it from, or which other hands it might have passed through. There is nothing else particularly noteworthy about the piece, which is one of the 13,000 pieces of Australian sheet music we’ve digitised to date.

The polka did not seem to make much of a splash even back in 1879. In May 1880, the Advertiser reports that the composer - ‘a young man of promise’ - has ‘found a home in Maclaren Vale’, that he has published several pieces of music including the Polka which has been favourably received by the press[5]. The Polka is described as simple, and in this piece of faint praise, that ‘the music is well suited to the words’. In 1937, the Advertiser, in its regular look back at the news of 50 years ago[6], included an excerpt about receipt of The football polka, dedicated, it notes, to the South Australian Football Association (all mention of Victoria has been dropped), and describes the piece as ‘simple and pleasing’.

Given that we have no record of any other published compositions by W. Geddes Taylor, and that this publication is held by only two libraries – the National Library of Australia and the State Library of Victoria – I suspect the polka may have been at the utmost limit of his compositional powers. It seems likely that this particular piece of Australian musical history may have gone unplayed for at least a century. Our copy is a bit grubby, has a few losses, and its spine has been bound with tape – but there is absolutely no sign of any fingering marks, which makes me wonder whether our physical copy was ever actually used by a real live pianist.

But on Friday 27 September, the National Library’s Trove team tweeted a link to The football polka, by W. Geddes Taylor, in honour of the upcoming AFL grand final. And within a few hours, this is what happened:

You may have gathered by now that rather than speaking about the impact of technology on the National Library and how it goes about its work, I would prefer to start with our user, Wendy Davis[7], and trace back through everything that needed to happen behind the scenes at our end - before 27 September 2013 - for this little tableau to unfold.

First, we needed to have a human being responsible for creating links between current events and items in our collection and then tweeting them to the Twitterverse – and of course for Twitter to exist.

In this case, we are talking about a member of the relatively small Trove support team, led by Tim Sherratt. This support team of 6, sometimes 7 people is itself supported by the Trove IT team – another 3-4 people at the best of times. So, a total of 10-12 people to build Trove’s unique collections, imagine its future, build its databases and architecture, keep it running, respond to its users, and think about ways to get Trove content out to the Australian people and beyond, including via Twitter.

Trove has more than 5000 Twitter followers, and we aim to tweet most days about items of interest in the national collection that Trove represents, and to keep our users up to date.

The Library’s main twitter account has more than 16,000 followers, and the Library’s full-time social media coordinator, Liam Wyatt, thought of asking what this piece of music sounds like. We collectively get retweeted quite a lot, so between the two accounts, we probably ended up getting this piece of Australiana in front of the eyes of a very large number of people, many of whom would never have heard of Trove, never heard of the National Library, and certainly never thought about whether any pieces of sheet music had ever been published to celebrate what I consider to be the only truly beautiful game. I should mention that as of mid October 2013, the combined NLA/Trove twitter accounts made us the fourth most followed Australian Government organisation, and the most followed cultural institution[8].

But using Twitter and other distribution mechanisms is not about competing for market share, about having more followers or more retweets than anyone else. It’s about getting our collection material into our users’ online spaces. That includes Twitter, Facebook, Instagram, Tumblr, and it will include new social media sharing networks as they emerge. It certainly includes Flickr Commons, where a total of around 500 National Library images – grouped into curated sets - have been viewed more than 1 million times.

This is really about two things: freeing our content by using technology to make our collections more accessible than ever before; and freeing our users from the expectation that they will walk through either a physical front door to our reading rooms, or a virtual front door called a website.

On the first point, I think it’s worth saying that the Library’s ability to leverage these new online opportunities is based on decades of investment in describing our collections, and in industrial scale, standards based systems that allow us to share our descriptions – machine to machine – all over the world. For Wendy Davis to find this piece of music on Twitter, we needed to have acquired it, catalogued it, digitised it – and then freed that content from its institutional backyard.

Like most big libraries, the National Library invests heavily in its Integrated Library Management System. Because we describe our collections in standardised ways – both in terms of content standard and machine readability – our catalogue records all flow through to Libraries Australia – the front end of more than 30 years of national collaboration to ensure that Australians know what is held in Australian libraries and how to get a hold of what they want.

From Libraries Australia they flow through to OCLC’s WorldCat, the largest aggregation of bibliographic metadata in the world, from where all sorts of interesting linked open data experiments are happening.

From Libraries Australia those records also flow through to Trove, which is indexed by Google. Google loves size and pays attention to large datasets, so Trove results tend to float to the top of many Google result sets. We do not expect our users to visit the National Library’s website, or to visit the Trove website or to even have heard about the National Library to discover our collections.

Three-quarters of our users find the Library content they want by searching Google and finding us. Although we still invest quite a lot in our websites and especially in meeting accessibility standards, we no longer see them as our users’ starting point – they are really just a route through to content which might meet a need a user knows they have or – in the case of Wendy – serendipitously spark a little piece of creative magic that brought this silenced piece of music back to life, even if just briefly.

The investment we have made in both our local cataloguing systems and our national discovery systems really started to pay off when we and others started digitising, and collecting born digital. Those databases are the backbone of both our local digital delivery systems, and our capacity to make the digital content of others discoverable.

Like everything else in the technological world, our delivery systems cannot stand still. I have been at the Library long enough to have seen the development of our first digital delivery systems, seen them age and start creaking, will be around to see the redevelopment of all our all delivery systems as part of our current Digital Libraries Infrastructure Replacement project, and expect to be around long enough to see even those age, get creaky and need re-engineeering. The pace of technological change means we can never stand still, that we need to plan our developments for quite short timeframes, and that what looks new, shiny and delightful to our users today will start to look tired tomorrow.

We have also invested heavily in our digital collecting and digital service systems. We’ve been collecting material in digital format for 20 years. We were the first National Library in the world to think it was important to collect Australian websites for posterity. There were no models for how to go about it. No tools. No standards. And certainly no new money. But we had a go, experimented, developed a tool, developed a service and along the way became experts in many aspects of web archiving. Which means that when we started to get a small flood of anxious questions from various Commonwealth Departments as to whether we had archived their websites before the September 7, 2013 election, the answer was yes.

We do not have the legislative framework to allow us to collect the Australian web domain – but we have done it anyway. We have completed a series of whole domain harvests since 2005, totalling 570 Terabytes of data. We do not yet know how to index it to make it useful, we cannot provide access to it because current copyright law does not allow us to do so – but at least we have it. We have trusted that the legal and technical frameworks that would make this content useful will be there in the future – and we are confident that they will be.

Collecting web archives is just one example of our digital collecting. The world is creating in digital and physical forms, so we collect in both forms, whether that is books, journals, newspapers, pictures, personal archives, maps or audio. In some cases, what we mean by ‘collect’ has changed, and this is a real change to our stewardship role. In the paper only world, we collected what our users need, took responsibility for preserving it, provided access to it in our own Reading Rooms, and either sent it by mail or sent portions of it by mail to distant users. In the digital world, we do indeed collect and take long term stewardship responsibility for material. But for a portion of our collection – the non-Australian electronic portion – what we offer is access rather than long term stewardship. If we stop paying our subscription, we lose access. If the company goes bust and has not made provision for preserving its content, there is not much we can do about it. We focus our resources on preserving Australian content.

Is this difficult? Yes. Is it expensive? Yes. Do we have a myriad of challenges to solve around collecting, preserving and providing access to digital content? Yes. Are we adequately funded to meet all these challenges? No. Do we have all the skills we need to meet these challenges? No. Why do we do it? Because the alternative is no alternative at all. If we only attempt what is easy, what is doable, what others have done before, what we already know how to do, or what has been specifically funded, we will rob our future generations of their future Football polkas.

Our physical format collections are vast, and we estimate that we have so far digitised only 3% of our collection. But along the way we have developed and shared digitisation standards so that individual organisations do not have to re-invent the wheel. We routinely show and talk about our digitisation workflows, again so that others do not have to re-invent the wheel. This is especially important in relation to our industrial scale newspapers digitisation program which – so far – is the biggest in the world. It is also based on more than 20 years of collaboration around the Australian Newspapers Plan.

That national collaboration means we have a truly national newspaper digitisation system, that almost all digitised newspaper content is aggregated into one place, delivering massive efficiencies. This is in stark comparison to other countries where small boutique projects – based on towns, or provinces, or states or languages – make it very difficult for our colleagues at projects such as Europeana or Digital Public Library of America to contemplate delivering anything quite as useful as what we’ve managed through the National Library. At the moment, we are realising that other similar content – such as Government Gazettes and Hansards – may never offer the same opportunities, because there have been a plethora of individual digitisation projects, rather than a national approach to content that’s of national interest. A National Digitisation Plan would help!

At the National Library, we are long past our boutique or cottage industry past, and the industrial scale of our physical collecting (around 5 semi-trailers of physical material still come into the building every year) is mirrored by our industrial scale approaches to the digital.

This covers all aspects of our work, not just the obvious ones of collecting and providing access. Over the last five or six years we’ve moved from having a myriad of non-standardised options for reference and copy services, to a streamlined approach in which every part of the Library uses the same reference management software, and the same highly automated copy ordering system.

 

That streamlined approach goes beyond the Library. National and State Libraries Australasia (NSLA) libraries all now use the same reference management software, and that means that we are just a whisker away from being able to seamlessly push reference enquiries between Trove and between each of our libraries all around the country. We will be using the same software to offer Trove users a consistent look and feel when they are ordering copies from our NSLA colleagues, most of whom do not have an ordering system as sophisticated as ours. In the NSLA sphere, technology supports our desire to offer the most seamless set of services possible to Australian citizens, who really should not have to figure out local rules and approaches with every library or archive or museum they approach.

But it is equally as important to recognise and seize less industrial sized opportunities as the technological market makes them available, including opportunities only 140 characters long. The National Library started using Twitter, and Facebook, and other social media mechanisms early. We did not write policies or rules first – we just got going, kept an eye on what was going on, and then later codified what was already a healthy self-regulating system into our Social Media policy.

One of the really great opportunities technology has given us in the last few years is the ability to free our content. In this image above, you can see Wendy Davis’ iPad sitting on her piano, but you probably cannot see that Wendy did not use the Library’s main digital delivery system:

Instead she used Forte, our sheet music app.

The interesting thing about this app is that we did not build it ourselves– at least to start with. The developer, Jake McMullan, also known as ‘Stripy socks’, found the Library’s sheet music content that was available through an open government data portal, and decided to experiment. Along the way, he thought we might be interested in what he was doing. And indeed we were so interested we supported him to complete development, and launched Forte in the app store ourselves.

That was just the first of many. We make almost all Trove content available via an API, where it has been used by GovHack participants and developers across the nation and the world. We expect that developers will make many apps, or small tools, or develop bots based on Trove’s content. We encourage them to do so. In fact, a small army of bots: are now serendipitously delving into Trove, or linking Trove content to content from other online services, with no human intervention. We are completely comfortable with this re-use of our content. Our next endeavours will be to make a variety of our services, for example the Trove annotations service (which supports tagging, comments and text correction) available in the same way for re-use, by whoever wants to use them.

So, back to Wendy Davis. Many, many things had to happen in order for Wendy to find our piece of sheet music and share a recording of it with the world. But to see those things as just about technology, or to just see technology as ‘impacting’ on our work would be very narrow.

So, I’d like to finish by leave you with a more rounded picture, with five words beginning with C that represent five things needed to support the role that technology plays in the work of the National Library, and that made Wendy’s experience possible.

  1. Computing technology. Yes, investing in technology is important. Around 10% of our Library staff work in the IT Division. At last count, we had 82 dedicated application and infrastructure servers, and 12 virtualised servers supporting an additional 58 application and infrastructure virtual servers. We had 1300TB of online storage and 1200TB of nearline storage. Our wide area network consists of a 100MBs link to the internet and we will have to increase that this year. We have WiFi everywhere for staff and users. Our staff use more than 100 software packages to do their work.
  2. Content. We all have fabulous content, and this is an asset we need to put to work for the good of our fellow Australians and of the world. Our colleagues at Digital New Zealand have an admirably simple statement about their mission. It is ‘We aim to make New Zealand digital content more useful’. Quite.
  3. Context. Not the archival context although I am as much a fan of that as most of you are, but the contexts in which our users are operating. We need to get our content into their contexts, and collect their content in ways that suit them, rather than expecting them to have everything mediated through our eyes.
  4. Collaboration. Australian libraries have collaborated at a national level for decades, and the benefits of that collaboration show. This is not a given in the library world; it is something precious and almost unique here in Australia. And I like to think that that’s because of the foresight of so many of my predecessors. We are really reaping the seeds they sowed twenty and thirty years ago.
  5. And last, Culture. Organisational cultures that are open to new ideas, open to freeing their content, willing to have a go, willing to make mistakes, and willing to just jump in and try things because they might work are the ones which will best surf the digital world we are in.

It is all of these things combined that meant that Wendy Davis could bring a not very good polka, written about a beautiful game by a man who has faded into history, back into some kind of present. And I think that’s worth doing.

[1] Nesbit, Paris 1896, Lecture on woman suffrage, delivered November 1895, at St. David’s Hall, Burnside, E.P. Nesbit, Adelaide.

[2] 1929, ‘MR.E. QUINTRELL DEAD’, Barrier Miner (Broken Hill, NSW: 1888 – 1954), 24 January, p. 3, viewed 5 December 2013, http://nla.gov.au/nla.news-article46048373.

[3] 1906 'BURGLARY AT KADINA.', Barrier Miner (Broken Hill, NSW : 1888 - 1954), 5 May, p. 4, viewed 5 December, 2013, http://nla.gov.au/nla.news-article44499318

[4] 1887 '[No heading].', Wallaroo Times (Port Wallaroo, SA : 1882 - 1888), 8 October, p. 4, viewed 5 December, 2013, http://nla.gov.au/nla.news-page10442013.

[5] 1890 'THE AUSTRALIAN NATIVES.', The Advertiser (Adelaide, SA : 1889 - 1931), 28 May, p. 7, viewed 5 December, 2013, http://nla.gov.au/nla.news-article26753184

[6] 1937 'FIFTY YEARS AGO.', The Advertiser (Adelaide, SA : 1931 - 1954), 31 May, p. 19, viewed 5 December, 2013, http://nla.gov.au/nla.news-article41648185

[7] Wendy Davis has kindly given permission for her story to be used for this paper. She has also recently recorded a Christmas song from the collection, available at: https://twitter.com/wendyldavis/status/410997867793244160

[8] At December 2013, the National Library’s combined accounts take us to third place among Australian Government organisations.