How to cause a crash

How to cause a crash
Testing the limits of Trove 7 to deliver a better product
1 October 2015

Planning

Trove is a community, so when the decision to go ahead with Trove 7 was finalised, one of the first concerns raised was how we could best engage YOU in the release process.

Web designers will often run what is referred to as a beta test. A beta test is designed to identify bugs, explore functional limits, and generate interest amongst the community of users who will engage with the new platform. Typically this might involve repeat users, who have a high stake in the product performing well, as well as those with a more casual interest.

We decided to go ahead with our stress test to engage both the high stake users and those who justed wanted check out the coming changes. Our aim was to engage as many users as possible so that we could properly test the limits of our platform as well as give all of our community a chance to see Trove 7 in action.

First controlled flight in Australia, made by Harry Houdini in a Voisin biplane at Diggers Rest, Victoria, 1911 [1] [picture]

First controlled flight in Australia, made by Harry Houdini in a Voisin biplane at Diggers Rest, Victoria  1911, nla.obj-144639380

Engagement

The decision to go ahead with the test involved a good deal of discussion around logistics. Having a reliable information page and a broad range of advertising channels were required to reach potential participants.

First we needed an information page that would be stable enough to support access requests from a large influx of users as well as being easy to understand and edit. For this purpose we used the News section of the Trove Forum.

The Forum is easy to update and is displayed through the newsfeed on the Trove homepage.

 

Most of our social media communication is conducted through the @TroveAustralia twitter account so we decided to announce the stress test there first. Twitter also has an exceptional reach if your aim is to engage casual users.

We tweeted our tweet:

Re-tweeted our tweet:

And then tweeted again:

The National Library’s Facebook page was also utilised to great effect.

Help us break Trove! At midday on Wednesday, 23 September, we are conducting a one hour stress test on some redeveloped...

At the beginning of the project, open communication channels were identified as integral to the success of our test. Potential users were asked to submit questions, report issues, and share their experience with other users through the @TroveAustralia twitter account.

You hit us with some very helpful queries:

And constructive feedback:

There was a bit of a stuttering start to the test:

But we got there in the end:

Monitoring

The Trove test environment was open from 12pm–1pm on 23 September. We had four servers running to deal with the load (usually we have 12 but we REALLY wanted to break something) and after a slight hiccup everything was up and running. In the first minute, Trove 7 was swarmed with eager testers, correctors, searchers, and a lot of people who just wanted to check it out. After just ten minutes the servers started to creak, as response times to queries spiked dramatically. Despite the slow response times you stuck with us and helped us highlight some fascinating occurrences.

You found some bad stuff:

You found some good stuff:

And some weird stuff:

Overall you seemed pretty happy with the new design even if you would have liked the test system to run a bit faster. The data we collected was priceless, encapsulated by this fantastic/awful spike that showed the slowdown in search times:

The video below shows what your testing looked like from an administrators' perspective. We’ve compressed the hour down to three minutes. The dots travelling towards the paddle are requests, while the paddle is our server. When the dots bounce back you are receiving approval to connect to the page you requested.

Three things we learned

  1. There was a failure in the database access layer, resulting in some odd error messages. This issue wasn’t expected but now that we know it can occur we will work to rectify the problem.
  2. Our search index stood up to a lot of punishment. Our IT boffins were particularly happy with the way it handled your requests despite the interface slowing to a crawl.
  3. The Trove 7 User Interface is shaping up well—we received some very positive feedback about new browse functions in particular. We always knew they were good, but it’s nice for someone else to agree.

Thank you to everyone who got involved, we’ve learned a lot about the way Trove 7 will function under pressure. Your help means we can fix, tinker and plan to provide you with the best product available.

Finally we really enjoyed talking to you throughout the process and have collected our favourite tweets into a Storify list for your enjoyment:

Comment: 
Looks like fun-sorry I missed out!