Staff Papers
Top Performing Interlending Operations: Results of the Australian Benchmarking Study
A paper presented by Tom Ruthven, National Library of Australia, at the Interlending and Document Supply 7th International Conference 15 October 2001 in Ljubljana, Slovenia.
Tom Ruthven
Susan Magnay
This paper presents the results of the major study of interlibrary loan and document delivery (ILL/DD) operations in Australia. The study has identified what makes these operations effective in delivering services to users and how to make these operations efficient. It is largest ever study of interlibrary loan and document delivery operations in Australia. It provides data on which to radically and rapidly improve the services. The report of the study will be released this month.
The paper is divided into four parts. First is a description of how the benchmarking study was undertaken. Secondly, there is a brief examination of the Australian library scene. Next, the results are detailed, and finally an outline is given of how the results will be used to improve services and activities in Australia.
1. Description of the study
Objectives of the study
The National Resource Sharing Working Group undertook the study with the assistance of the National Library of Australia. The Working Group was established to ensure a robust, reliable and cost-effective interlending system for Australian libraries and their users.
The Group had a simple objective for the study: to discover what the high performing document supply units were doing that the rest of us could learn from. In other words, how to improve services to users and reduce costs.
The three key objectives of the study were to:
- identify the characteristics of high performing ILL/DD operations,
- be an instrument for raising awareness and changing ILL/DD practices, and
- provide a standard set of data to assist libraries to benchmark their operations.
Methodology
The study instrument was based on that used in the Association of Research Libraries study, Measuring the Performance of Interlibrary Loan Operations, undertaken in the United States with North American research libraries. The Australian study was broadened to include the special library and public library sectors, the state and national libraries, as well as university libraries.
Over 100 libraries across Australia participated, including 30 special libraries, 32 of the 37 universities, 20 public libraries, all the state and territory libraries and the National Library.
The survey instrument had four components.
The first part of the instrument was a detailed questionnaire asking information about the general characteristics of the library and the ILL/DD unit. The questionnaire was 17 pages long.
Costs were collected in the second part. The survey asked for actual costs, including many from the last full financial year. The price of equipment purchased over the last three years was requested so that these costs could be amortised. The cost data collected included:
- staffing,
- network and communications costs,
- delivery charges,
- photocopying and scanning,
- office supplies,
- equipment,
- software and maintenance, and
- the fees and charges each library paid and payments they received.
Each library tracked requests during October and November last year. They noted the time the user asked for an item, when the request was sent to each potential supplier, and when the user was notified that the item had arrived or that the request had been unsuccessful.
Finally the user was asked to complete a user satisfaction survey for each item they requested.
Analysis
The survey generated 25,000 pages of data that was keyed into spreadsheets. Each general characteristics questionnaire and cost sheet was checked to ensure the data supplied was accurate and comparable. A sample of the turnaround time and user satisfaction survey forms was checked.
The government statistician, the Australian Bureau of Statistics, was commissioned to analyse the results. The statistician undertook a correlations analysis to identify possible associations between variables using logistic regression modeling. 50 variables were included for requesting activities and 42 for supplying activities. The statistician identified strong and weak variables that were significant in explaining efficiency.
In particular the Statistician was asked to look at turnaround time, fill rate, and unit cost for requesting activities, and fill rate and unit cost for supplying activities.
The National Resource Sharing Working Group looked at the implications for the variables that were found to be significant. The weaker correlations were checked to determine if they indicated other possibilities for efficiency, and the characteristics of those libraries that were identified as high performing were examined in detail as well as those identified as low performing to highlight any differences.
2. The Australian interlending scene
These brief notes will help place the results of the study into context, particularly as many of the results refer to automation as the key, and the use of up-to-date online union catalogues.
Australia is a large country where most of the 19 million people live in 8 major cities. Libraries tend to be concentrated in the major centres but there are significant research organisations and universities outside the major cities, as well as public and school libraries everywhere. Document supply services are usually valued parts of these libraries.
Australia is fortunate to have had a long history of building comprehensive union catalogues with online retrieval services to access them. These include national, sectoral and regional networks. Some examples are the National Bibliographic Database run by the National Library with over 12 million cataloguing records and 37 million holdings from 1,600 libraries around Australia. It is accessed online through the National Library's Kinetica service. It has a web search interface and an automated interlibrary loans module. ILANet is run by the State Library of New South Wales and also has an online interlibrary loan service. Health libraries run a very successful consortium with an online serial list and an online interlibrary loan service. The State Library of Western Australia runs an online union list for all public libraries in that state as well as an automated interlending service.

Australia's size compared to Europe
Distances are very large. Australia is as large as Europe. If Perth is in Spain, then Sydney is in Turkey, the southern most point of Australia would be in North Africa and the northern most point in Norway. Moving library material around this vast area has resulted in the adoption of many electronic mechanisms. For example, 75% of requests sent to the National Library are received electronically and 50% of the material sent is scanned and sent over the Internet. There are efficient mail systems which can deliver items across the country usually within 3 days and within 24 hours for a premium price.
Australia has a great diversity of libraries. The National and State libraries contain substantial collections of material relating to Australia and the region. The universities range from those over 100 years old to more recent universities established in the 1970's. Public libraries can be well funded with extensive local resources to small outposts in the remotest areas of Australia. Most government departments and research organisations and many private organisations have libraries, often having significant collections in their field.
The ILL/DD services in Australia try to meet the needs of clients in all types of libraries. The study has shown that although each sector has a slightly different emphasis in service delivery which best suits their client groups, the performance levels within a sector vary considerably. However, the key measures to improve services are the same across all sectors.
Key measurements
As mentioned above, the key characteristics measured were:
- turnaround time,
- fill rate,
- unit cost, and
- patron satisfaction.
Unit cost, fill rate, turnaround time were examined for requesting activities. Only unit cost and fill rate were examined in detail for supplying activities. The limited data collected for supplying turnaround time was analysed to draw some general conclusions about this measurement. The patron satisfaction survey was not seen as a detailed enough tool to draw more than general observations.
3. Results of the study
Each of the performance measures are examined in turn, including what was identified as making a top performing library. It is worth noting that no library came in the top 10% on all categories, so all libraries in Australia can learn from these results.
Turnaround time
This paper examines the turnaround time for requesting activities only, i.e. how long it takes from the time a user asks for an item to the time the user receives the item or is told that it is unavailable. The report will include information on supplying turnaround time.
Graph 1 shows that the 10th percentile library had a turnaround time of just over 7 days whereas the 90th percentile library takes almost three as long at 19.1 days. The median library takes just over 10 days.
The medians for each sector are shown, with special libraries on average having the fastest turnaround time and public libraries the slowest. This reflects the emphasis special libraries place on speed, whereas the users of public libraries tend to be more concerned about cost and less about speed.

Graph 1: turnaround time
In any case, there are libraries in each sector that are two to three times quicker at getting material than others. So what are these fast libraries doing?
The key factors mostly relate to automation, and automation covering every facet of the ILL/DD process:
Most of the users submit their requests electronically from their desktop.
ILL/DD staff process the request on the same day that it is submitted.
ILL/DD staff search large union catalogues online. This includes the National Bibliographic Database, online union lists of serials, and combined catalogues of the libraries most request from, such as the OILLS system for public libraries in Western Australia.
The user's electronic request is sent to the first potential supplier without rekeying any information.
The item is delivered electronically wherever possible, such as using the Ariel software.
The user is notified as soon as the item arrives, also electronically, and if the item has arrived electronically sending it to them or notifying them by email that it can be picked up electronically from a system such as Prospero.
Another factor is choosing the first supplier carefully. Speeding up part of the process is under the requesting library's control and remarkable reductions in turnaround time can be achieved in these areas, but the supplier's turnaround time is out of the requesting library's control. The quickest turnaround times were achieved by libraries which had most of their requests filled by the first supplier. The choice of the first supplier goes beyond just looking up who holds an item, although this is important. It also meant that staff are trained and have expert knowledge of other library collections and policies, being able to look at whether an item is available, maintaining good relations with major suppliers, and supplying information that makes a supplier's life easier.
Fill rate
Fill rate for requesting operations informs us of the success in finding an item, whereas for supplying it tells us how effective a supplying operation is at stopping requests coming to them for items they do not hold or cannot supply.
Graph 2 shows the results of the study for requesting. The library at the 90th percentile was able to fill requests 99% of the time, and the median library was able to get 92% of the time. However, for some libraries this dropped to 77%. The median for each sector is shown with special libraries having the highest average fill rates.

Graph 2: fill rate for requesting activities
The study found that there are three factors which enable a library to get an item 99% of the time. Firstly, you need to search union catalogues. This, amongst other factors, is becoming a theme for efficiency and effectiveness. One thing to note is that the high performing libraries in this criterion used union catalogues that were comprehensive, up-to-date and easily accessible. There is a fall in fill rates when catalogues that do not have potential suppliers' entire serials collection, are not being updated regularly, or that the search or display options do not allow information to be found easily. For example, printed lists which are circulated, holdings statements that do not include missing issues, and serial runs without start and/or end dates of a library's holdings.
Secondly, you need expert and effective searchers. You need to have trained staff who know what they are doing.
Lastly, the more permanent staff you have the better the fill rate. This emphasises the need to have trained and expert staff, people who know that they are responsible for delivering a good service over a long period.
Perhaps more interesting is the fill rate for supplying operations shown in graph 3. The 90th percentile library was able to supply 97% of all the requests it received, whereas the 10th percentile library could only fill 66% of its requests. One of the major findings of the study is that a higher fill rate as a supplier is associated with reduced supply costs and faster turnaround time. Minimising requests for items that are not available eliminates the work required to say you can't supply an item. In addition, the requesting library can provide a faster service to their users as they don't have to go to a second or third potential supplier where the supplier cannot supply an item.

Graph 3: Fill rate for supplying activities
There are two key factors which increase fill rate, both relating to what you tell the rest of the world about your collection.
First is the use particular types of cooperative agreements, those where a database of holdings is kept very up-to-date and lending policies are more liberal.
Second is the addition of holdings to the National Bibliographic Database and keeping them up-to-date. Whichever way the data is examined, those libraries that told everyone what they hold had cheaper costs, more efficient processes and more effective services.
A slightly less important factor is the number of times that the shelf is checked for an item. This activity had no effect on efficiency. If the item is not on the shelf the first time it is checked, it is unlikely to be there the second time. However, a second check extends the time that the requesting library discovers that another library must be asked for the item, increasing the time the user has to wait for the item.
Unit cost
The study included a wide array of costs to calculate the unit cost. This paper examines costs for requesting only, i.e. how much it costs to request an item.
Graph 4 shows the average unit costs of each component of requesting an item. The average overall cost was AUD$32.09. Public libraries had the cheapest costs, again reflecting the priority of this sector. The public do not want to pay much and are less concerned about the time an item takes to arrive. The National/State library sector is not shown but is included in data used to produce the combined figures.
Two costs contribute 85% of the total cost: staff and fees. Efforts in cost reduction should focus on these two areas where the greatest saving can be realised.
Each staff member must process more in order to reduce staff costs. Those libraries in the study that had the cheapest cost staff costs always had other high performance factors, i.e. an efficient service results in staff being able to process more which in turn reduces unit costs.
Fees are not in the control of the requesting library but can still be reduced by being part of a cooperative agreement where there are no fees. Although there is also no income for items supplied, there are flow-on cost benefits such as not having to raise or acquit invoices, not having to track payments, and not having to monitor budgets.
One final thing to note with unit costs is that equipment, network and delivery costs make up only 10% of costs. Automation, although often expensive at the time of purchase of equipment or software is not a major contributor to cost in the long term. Costs can be reduced by investing in technology to make your staff more efficient.

Graph 4: Average unit cost for requesting an item
Patron satisfaction
Users in all sectors are very happy with the services being delivered with over 95% expressing a high level of satisfaction. However, comments on the patron satisfaction form indicate some concerns.
Quality of copies was an issue, including missing pages, blurry copies and illegible diagrams and photographs. Many mentioned they want the item delivered electronically for improved quality, improved delivery time and as being more convenient.
Another set of concerns related to the lack of information they can get about a request. Users want to be able to track their requests themselves to see how their request is progressing. They also want a more detailed explanation when an item is not available. The library which is unable to supply an item should consider more detailed messages so the requesting library can inform their user fully. Only saying 'not available' is not useful to a user.
Users were asked how much they would be willing to pay. The average a user was prepared to pay was AUD$5.28, even though the average charge to users was AUD$15.03. It is noteworthy that many users thought ILL/DD should be free as they saw it as a core service of their library.
4. Improving services using the results of the study
The study has shown that changes in five areas will result in dramatic improvements to the effectiveness of ILL/DD operations and increase their efficiency.
a. Examine workflows to ensure there are as few steps as possible involved in the ILL/DD process
The most efficient libraries had the least number of steps involved in obtaining an item. This may involve patrons creating an electronic request rather than manually filling out a form and having someone key the information into a system. It may be aiming to have over 90% of requests filled by the first supplier by training staff in searching techniques and using union catalogues. It may be adopting electronic notification to users when the item is ready to collect rather than transferring that information to another system.
b. Implement an automation package or include an ILL module in the next tender for a library system
It is clear under every performance criterion that libraries need to automate their ILL/DD processes to become efficient and effective. Automation literally saves days in turnaround time and reduces unit costs for requesting and supplying.
Libraries can improve their requesting performance by:
- having the patron create the request;
- seamlessly transferring the request to a supplier;
- use a service such as the Kinetica Document Delivery Service or GRATISNET to automatically route the request;
- notify patrons of the results of their requests electronically; and
- deliver copies electronically to the users desktop.
On the supply side, libraries can improve their performance by:
- receiving a majority of requests electronically; and
- using a system that facilitates easy electronic tracking of the progress of requests.
Investment in automation is an only a marginal cost compared to staff costs. A relatively small investment in automation can lead to large gains in effectiveness. It is critical to recognise, though, that to maximise the effect of using an automated system, workflows need to be examined in detail to ensure that all processes are streamlined.
c. Ensure ILL/DD staff are well-trained in the resources and systems used
Staff is the major cost of most ILL/DD units. Maximising their efficiency and competency can make the greatest savings and improvements in the effectiveness of an ILL/DD service. Investment in training staff to make them expert searchers and knowledgeable about key resources will contribute to a faster and more effective service. It may also provide opportunities for libraries to expand the services they offer without increasing their unit costs.
d. Add and maintain holdings on union catalogues
This was a key factor for high-performing libraries. It is clear that libraries are more efficient suppliers if they generally receive requests for items they hold their fill rate is higher, which minimises the staff time spent on processing requests for items not held. Libraries are also more effective requestors if they have access to up-to-date information about other libraries' holdings it reduces staff time spent searching for source libraries and makes it more likely that only one supplier will need to be approached to fill a request.
This requires a commitment by individual libraries to make accurate and up-to-date holdings information available about their collections. It also requires commitment and co-operation from the library community to encourage all libraries to contribute to the ILL/DD system.
e. Investigate options for cooperative agreements with key requesting and supplying libraries
There is substantial evidence that having a well-regulated cooperative agreement assists efficiency and effectiveness. The most effective and efficient cooperative agreements had the following characteristics:
- no fees;
- holdings easily available online and other collections are well known to members;
- a joint electronic system is used to enter and track requests;
- most requests are filled by libraries within the agreement;
- transaction numbers within the agreement are fairly high and are fairly distributed; and
- members cooperate and share information to ensure the system works smoothly.
Benefits include reductions in administration costs and a greater availability of material for users where there are more liberal lending policies in place between libraries with an agreement.
Response to the study
In response to these outcomes, the National Resource Sharing Working Group will be undertaking a series of initiatives.
Training in best practice
Training courses will be developed for ILL/DD practitioners, focusing on improved work practices, use of tools, and how to obtain knowledge of collections.
Seminars to discuss policy issues
These will be aimed at Chief Librarians and senior executives in organisations to discuss the efficiencies that can be achieved and how to achieve them.
Identify and promote best practice
Best practice will be facilitated by developing model workflows, promoting the development and implementation of ILL/DD software development, and providing information on equipment.
Encourage the examination of operations
All libraries in Australia will be encouraged to benchmark their ILL/DD operations against this national and sectoral set of data.
