Staff Papers

Evaluating the public library portal

 A paper by Roxanne Missingham , Resource Sharing Division, National Library of Australia ; Fran Wilson, Collaborative Resource Sharing Services, National Library of Australia ; Lynn Hedley , Kinetica, National Library of Australia ; and Janet Smith, Southern Tablelands Regional Library Service

Introduction

Australians are avid information seekers and consumers of information in a great variety of forms. Their search for information now includes the Internet for almost all of their regular “discovery” activities. Libraries are increasingly providing services which enable Internet access to services and collections – whether they be catalogues, subscribed services and/or electronic journal collections and reference services – to meet the growing needs for convenient access by users. For the general public the National, state and public libraries have increasingly provided access to digital content, collaborating on purchasing and moving to collaborate on access services.

Over the past 14 months the National Library and a group of public libraries have been engaged in a pilot project to deliver better understand the needs of Australians to access information resources. The challenge for the National Library and public libraries is to deliver access to our collections and services in a manner that is easy to use, relevant and timely for Australians. Collaborative projects undertaken in the past, such as PictureAustralia and the National Bibliographic Database, have concentrated on providing access to defined sets of resources. The move to electronic service delivery in public libraries and the National Library has enabled us together to better understanding of user needs. This, combined with the maturity of technology, provided a basis for discussions in 2002 and 2003 to investigate a more integrated approach to information access for Australians.

After considerable discussion with public libraries, a pilot project was initiated to develop a new service, Information Australia, which offers a single “Googlised” approach to a range of Australian print and electronic resources. This paper outlines the evaluation of the pilot portal, including policy and technical issues. It summarises the evaluation activities of the partnership between public libraries and the National Library to develop a service which makes library collections accessible and relevant to the needs of Australians in the online environment.

Scope of Information Australia

The National Library’s involvement with public libraries to deliver resources directly to Australians originates from the strategic directions.

“Our major undertaking in 2003 - 2005 will be to provide rapid and easy access to the wealth of information resources that reside in libraries and other cultural institutions and to break down barriers that work against this. Services supporting access to library information will be simplified and made more user-friendly, and will be widely promoted.”

To meet this broad goal, the National Library’s focus includes the investigation of services to overcome barriers and create access to electronic and print resources across the nation. The National Library’s fundamental approach to overcoming barriers can only occur through working with Australian libraries to meet the needs for all Australians.

Australian public libraries are a key service throughout the nation. With over 93 million visits a year Australians are primarily served by their local public libraries. Public libraries have increasingly offered access to electronic resources to their users – ranging from full text electronic services to homework help. Public libraries have also been key collaborators in the development of online service for their customers, including collaboration with the National Library in PictureAustralia and state based initiatives such as Gulliver.

For public libraries the goals of providing relevant online services which incorporate innovative delivery services and consortial purchasing to increase the range of resources available were particularly relevant to the project. Public libraries have developed many services to enable their users to access electronic resources. Kay Poustie, library consultant and Chair of the Library Board of Western Australia, has commented on the transformation of public libraries in Australia and the strength of their community access role ( Leyland, 2003). Public libraries have a strong history of collaboration. The portal enables the connections between these libraries to develop further.

The partnership resulted in an agreement to pilot a service, Information Australia. The concept developed at a workshop held with Yarra Melbourne, Yarra Plenty and National Library staff in mid-2002 to explore issues relating to supporting access to resources for the public. The workshop identified a need for a user-friendly interface that enabled simultaneous searching of several different databases. The portal aimed to enhance access to quality information resources online and available through Australian libraries.

The Information Australia pilot commenced in October 2003 and while it was planned to run as a trial project for one year, has been extended until June 2005. Evaluation commenced in August 2004. A detailed paper on the development of the pilot was presented to the 2004 VALA conference.

Framework

Information Australia is an initiative based on increasing access to information resources for Australians. The public libraries involved are Brisbane City Council, Yarra Plenty Regional Library Service (Victoria), Northern Regional Library Service (Moree, New South Wales), Thuringowa Library Service ( Queensland) and Southern Tablelands Regional Library Service ( Goulburn, New South Wales).

The overall objectives of the Information Australia pilot are to:

  • develop better access to Australian print and online resources for community across the nation through an Internet based portal;
  • develop a new business and funding model for access to these resources and the material from the National Library’s collection to support public library needs;
  • enable document delivery through direct access to online resources or via the Australian interlibrary resource sharing network; and
  • evaluate the service and the needs of Australians for access to online resources and collections in the nations’ libraries.

The three prime deliverables of the pilot are:

  • a working portal supporting access to a range of Z39.50 compliant data sources and interlending workflows;
  • research into the information needs of Australians including interface and content issues; and
  • the development of a business model applicable to a range of Australian public libraries.

Information Australia Portal

The Information Australia web site brings together a host of information services making them available from a single entry point. The portal uses the Internet to extend access to Australian research resources including:

  • the Australian National Bibliographic Database (ANBD), containing information on more than 35 million titles held in Australian libraries;
  • key Australian journals in full text via the Australian Public Affairs Full Text (APAFT) service;
  • PictureAustralia - providing integrated access to more than 1 million online images from 40 cultural institutions across Australia;
  • directories such as the Australian Libraries Gateway database, Maps Directory and Oral History Directory; and
  • links to other online services and information resources such as Mura Gadi, AskNow! and Australian Government Publications.

While there has been a focus on providing immediate access to electronic materials, Information Australia includes the ability to request materials through Interlibrary Loan services. In order to gauge the extent of public demand for Australian resources if there is no charge imposed, interlibrary loans and copies from the National Library’s collection are being provided free of charge for the duration of the pilot.

The resources available through the portal for which there is a charge, for example APAFT, are purchased on the basis of consortial arrangements, building on the work done on cooperative purchasing with National, state and public libraries through the CASL Consortium.

The target audiences and beneficiaries of Information Australia are Australians who are seeking information about Australia and resources in Australian libraries and related institutions. They include:

  • public library users (from the comfort of their own home using an authentication process, or within their public library)
  • researchers
  • students and teachers
  • recreational users
  • family and local historians
  • university students
  • local politicians
  • public library staff

Access to the trial was limited to the 5 trial participants. The front page of Information Australia is designed to be a simple, user-friendly “Google” type interface. Patrons simply type in a keyword into the search box and click submit. This contrasts significantly with the Kinetica search screens used till December 2004:

Comparison of Kinetica and Information Australia basic search screens

Comparison of Kinetica and Information Australia basic search screens

Figure 1: Comparison of Kinetica and Information Australia basic search screens

The portal distributes the query to multiple Z39.50 targets in parallel and waits for the results, which are formatted and sent back to the user as they arrive. Hence, results from Z39.50 targets which complete their search quickly are visible to the user even though some targets have yet to return their results. The upside of parallel and asynchronous searching is that the user gets some results quicker.

The “get” functionality is as follows:

  • Images in PictureAustralia are displayed in thumbnail which clicks through to the more detailed image on participants web sites;
  • Fulltext articles in APAIS (APAFT) click through to PDF files, with access covered through subscriptions;
  • Material which is listed in the Australian National Bibliographic Network as held in Australia libraries can be requested – either as loans or copies, with pilot participants offering reciprocal gratis interlending and material from other libraries most often available on a charge per article basis consistent with the Australian Interlending Code.

Figure 2: Requesting options for loans and copies are available from each bibliographic record in the Australian National Bibliographic Network

Figure 2: Requesting options for loans and copies are available from each bibliographic record in the Australian National Bibliographic Network

Patrons can create interlibrary loan requests online for ANBD and APAFT items. Bibliographic details are migrated into a form and patrons must enter their contact information. They can indicate if they only want the item if it is free or, if they are willing to pay for the item, how much they are willing to pay. Information from the ILL form is then parsed through to their local library’s Kinetica Document Delivery system using IP authentication. Interlibrary Loan staff then check the request and sort the libraries on the rota according to whether or not the patron wishes to pay for the item. They may add the National Library to the rota or other libraries with which they have reciprocal borrowing arrangements. The National Library or other supplying library will use Kinetica Document Delivery to ship the item to the requesting library. The National Library Document Supply Section can supply larger documents using Ariel web delivery.

Critical issues addressed through the pilot include:

  • supporting skilling of library staff to develop appropriate skills to market the service;
  • communicating information about the service to the general public;
  • understanding the needs of Australians and their expectations of access to resources online and through interlibrary lending; and
  • research into interface design.

Evaluation methodology

The public library portal was established to better understand how the Australian library network could meet the needs of Australians, be they in major capital cities with access to many collections or in regional Australia with access to their local public library. The evaluation process undertaken focussed on qualitative and quantitative indications selected to establish a better understanding of the relevance of library networks as well as the success of the portal.

Three major evaluation activities were undertaken. The first was to obtain information from users of the service in the mid term part of the pilot. This took the form of collecting information from the public libraries on the feedback and stories of use by their patrons and also through an online survey in mid-2004. This component of the evaluation was undertaken to obtain deeper insights into the value of the portal and the range of services supported.

The second evaluation was undertaken using a market research approach. An external expert was selected through a tender process to undertake qualitative research into the information behaviour patterns of library users and relevance of the pilot. This provided an opportunity to compare the experiences of those using the Australian National Bibliographic Network through their local systems and the Kinetica Search interface in an academic and state library context with users of the Information Australia portal in public libraries.

The third evaluation of the portal was undertaken with detailed analysis of the actual usage of the service for searching and requesting interlibrary loans. In this evaluation a comparison was made of the resources requested by public library users through the portal and the general usage/demand for resources from the National Library.

A summary of each of these evaluations follows.

Mid term evaluation: For this analysis case studies were compiled demonstrating the value of the portal and in addition a survey was undertaken through the web site. The case studies suggested that access to the portal supported a considerable increase in research undertaken by Australian using the public libraries participating in the pilot. Some libraries reported the local history and environmental groups had been able to take significant advantage of the resources available through the portal. Case studies also indicated the value of the project from the communities point of view.

The survey was based on using a pop up form on the web site with around 25 questions. There was a small response rate perhaps suggesting that the pilot was in an early stage or that the users of the service were focused on finding information and were well satisfied. The results of the survey indicated that Australians sought to use a wide range of resources through the portal:

Information sources sought through Information Australia

Feedback from participating libraries suggested that users found the fact that a single (federated) search covering all these materials/formats was very advantageous. The most sought after resources were books and journals articles, with the availability of an ordering services providing a significantly high value, particularly to regional and remote users.

Most users were searching for a particular item or subject (85% and 18% respectively). The high level of searching for a particular item may be a result of the fact that users had previously not had access to such a comprehensive resource discovery service. Personal research was the primary reason for searching (59%). Librarians in public libraries were more likely to use the advanced search option. A remarkably high percentage of respondents indicated that they had requested material through the portal (85%) and the majority of these (78%) found that it was easy to use the ordering service. Only 7% found ordering difficult.

Respondents were asked how much they would be willing to pay for an interlibrary loan or copy, to assess the possible impact of moving from the free reciprocal interlending service. 48% of respondents indicated that they would be not be willing to pay, however 55% (this was a multiple choice question) indicated they would be willing to pay between $1 and $10.

A very high satisfaction rate was reported – 87% were either extremely satisfied or very satisfied.

Satisfaction rate

Comments suggested that the service was very relevant to the information needs of Australians:

“This is a wonderful service that has enriched my life greatly. I feel deeply appreciative of access to so many wonderful books. I have learnt so much and personally benefited on so many levels from many of the books I have accessed. I am not wealthy, but through this service I don't need to be. I'd like to thank somebody. Could this thank-you please be circulated to people involved in this service.”

“It’s a great service - keep it, extend it, improve it!!!”

“I think it's great that if you can't find a book in your own library, you can use this as a tool to perhaps find it elsewhere. Very useful! Thank you for this great tool.”

“Information Australia is the next best thing to the creation of Internet. Working in an industry where literature is scarce, Information Australia is about the only known useful integration of databases I have come across. This should have been implemented a long time ago. I am so excited, knowing that the world's knowledge is accessible at the tip of my fingers. I would definitely recommend this continuation of this program, even on a fee paying basis. It is the best thing that ever happened!!!

Information Australia is a great service. If everyone knew about you and made use of you, you'd be overwhelmed. Thanks for the service!”

The area where the service did not meet user needs as successfully as it could have was in relation to the ordering service. Users found it confusing that some resources were available for free and some not, and that this information was not clear at the point of ordering. There were also some significant delays in the supply of material, mostly due to requesting of popular material for which there was a waiting list in the home library.

Focus group research: In October 2004 research was undertaken into the information seeking behaviour of Australians through focus groups and structured interviews in public, state and university libraries. Participants were able to discuss their general information behaviour and then focus on their expectations of access to the Australian library network. Public library users were selected who had used Information Australia. University based participants and independent scholars had used the Kinetica search service to access the ANDB, some through their local catalogue OPAC via Z39.50, others via the Kinetica Search interface.

The research identified some very interesting trends in information seeking behaviour:

  • most participants from the public and university sectors used the website and their library catalogues as first sources of information,
  • those in the university sector most often used their local library catalogues as the first point for searching, a smaller number used national catalogues and databases, such as Scifinder; and
  • Independent scholars also used websites and catalogues, however this group appeared to use a wider range of resources to search, perhaps due to the depth of their research.

Expectations of online access to information about library collections is now very high:

  • Public library users want to be able to find and request material online from where ever they are. They also increasingly are using the catalogue to identify material which they will purchase if required. One participant commented: “go straight to X Library, put an item on hold or place an order – but waiting list can sometimes be very long. Buy book if important.”
  • University participants expected access to Australian collections to be available in a simple and intuitive manner and found that the current process failed to enable easy access to the ANBD;
  • Independent scholars expressed considerable interest in federated searching and preferred to use a federated search service than single databases.

Interestingly the barriers perceived in terms of obtaining access to collection materials once they were found through the ANBD were seen as very significant. Charging for interlibrary loans and opaque interlibrary loan processes were cited by users in all three groups as significant barriers to accessing collection materials. Perhaps the strongest mismatch between resource discovery service and user needs was expressed with the following comment:

“Found one particular book that I was after and when I finally placed the order a message came back that I would have to pay $13 for the book – I found out the publisher had 500 copies of the book for sale at $15 and I could have bought the book – that was the last time I used it - the book came from the State Library and I got the impression it was a rare book and you had to use it for a set period - since bought that book.”

This is a very powerful indication that the traditional national union catalogue model of providing a resource discovery service with holdings information only, but not easy ordering paths to suppliers including libraries and commercial suppliers, is outdated. The two key themes emerging were that users of a national resource discovery service believe that access means “find” and “get” not just the ability to find where a publication is held and that for many “get” includes the option of borrowing from a library and also the option to order if a user wants to keep the publication.

When asked about the interfaces for searching the ANBD and access to Kinetica or Information Australia comments clustered around four main issues:

  • Content issues
  • Ease of use
  • Visibility and knowledge of the service
  • “Getting” issues.

Content was quite a critical issue for users. Most searchers expressed the view that federated searching was an essential value of the service. Many however were not clear about the coverage of bibliographic and full text resources. For many the service was the only federated search service they used, leading to more comparisons with Google that other federated services. Comments such as:

“holdings in libraries local to my area would be useful”

“more member library (collections) should be on the system”

suggest that the coverage of the service needs to be as comprehensive as possible to ensure the search results are as effective as possible. More users expressed a desire to have direct assess to full text resources than not, and some suggested other databases that should be included in the federated search service.

Ease of use was a topic dear to the hearts of many users, and increasingly users expect a Google-style approach. Users of the traditional OPAC style interface to Kinetica were quite critical with comments such as:

“Found it maddening at first, but got used to it “(Z39.50 access to the ANDB through a local system)

“A great thing cloaked in the worst sort of software and user interface”

“Kinetica is more effective in getting access to things held in libraries but Google is obviously more user friendly in terms of returning what is out there – using inverted commas, the ability to search by a phrase – if Kinetica could do that it would be great”

Users of Information Australia found the Google style interface was more intuitive and easy to use. Typical comments were:

“love the service”

“found it to be very quick and quite easy to use”

This confirms that a revolution has occurred in user expectations. While content may be King the search system is critical to actually encouraging use of the service – and Google is Queen in terms of style and effectiveness. Interestingly users commented that the system should be so intuitive that you don’t need online help of manuals.

Visibility and knowledge of the service proved to be the most significant limiting factor in use of the service. Many commented that they had been unaware of the service, that it was too hard to remember all the different paths to resources and that regular promotion and marketing should be undertaken to ensure visibility and use. The competition for attention in search services means that communication activities need to be more regular and timely than when the world was primarily focused on printed indexes and bibliographies. Users suggested that the service should be available freely on the Internet to enable Australians, wherever they were, to tap into the rich resources held by the nations libraries. Visibility of the service on websites was poor:

“Bookmarked it because I could never find it going through the library catalogue.”

“Getting” issues remained the most complex and challenging. Users of both services indicated that they found the complexity of the interlibrary loan ordering process, unpredictable charging and delivery times frustrating. Users of the standard Kinetica service commented:

“It would be wonderful if there could be some link from Kinetica and you don’t have to go back to the damn library thing and have to fill in the form.”

Those using Information Australia appreciated the fact that the service automatically transferred bibliographic details into the ordering form. Users identified a number of areas in which the service could be developed to be more effective. The lack of automated confirmation and regular automatic updates was frustrating. Worse however was the frustration of ordering and not promptly receiving material. Comments made include:

“I ordered a book and I still haven’t got a message about it for the last couple of weeks and I think that this is a bit of a problem…”

All user groups expressed a reluctance to pay for interlibrary loans – though when pushed they indicated that charges in the region of $2 to $10 might be paid. The standard charge for an Australian Interlibrary loan (not rush, express or priority) is $13.20 including GST. The gap between the expectations of users and our charge, which recoups administrative costs suggests that the potential to delivery this service direct to Australians may be limited. Public library users suggested that an appropriate turn around time from ordering to receipt by the user would be one to two weeks, while university users suggested that one working week would be appropriate. The Australian standard for the core service is five days processing time, with delivery to the user having additional delay due to postage.

Usage information: While the service was launched in late October in 2003, because of school holidays and staffing resources most libraries launched the service in early 2004. Communication with library patrons was achieved through incorporation in general library training sessions, articles in the local press and library newsletters, launches and in communication with groups such as local history societies and school libraries. Considerable interest was expressed by school librarians, particularly in access to full text Australian journal articles. Usage from December 2003 to November 2004 was:

InformationAustralia - total searching

Both simple and advanced search screens were considerably used. Over the first three quarters of use the trend was to increasingly use the simple search screen. The pattern of usage reflects to some extent the size of the library. Brisbane City Council Library is a very large library with many branches and high usage was expected. Yarra Plenty Library service also provides a service to a considerable population in northern Melbourne. The relatively low level of usage may reflect issues relating to communication and marketing.

Use of the web site was focussed on searching. Only 3% of website visits were to the “About” page, 2% to “Help” and 1% to “contact us”. This suggests that the interface succeeded in achieving an intuitive, easy to use search service because of the minimal use of online support documentation.

Many of the searches resulted in interlibrary loan requests for loans and copies. Again the pattern reflects the size of libraries in a general sense.

Loans and copies requested

The supply of material through request is a slightly different pattern because of the inclusion of the National Library as a supplier.

Loans and copies supplied

The total number of copies and loans supplied through Information Australia for the period November 2003 to November 2004 was over 4300.

The usage statistics demonstrate that Information Australia has succeeded as an easy to user service providing the ability to undertake federated searches through simple and advanced interfaces. The high usage of the requesting component suggests that the service met a significant need for the public library users in these communities. Analysis of the range of resources requested from the National Library found that 48% of requests were for Australian resources. This is considerably more that the average of 28% of regular interlibrary loan requests.

Older material was also more heavily requested than newer material. 91% of material requested from the National Library was more than 5 years old. This contrasts considerably with existing use of the service via libraries where approximately 80% of documents supplied are from material less than 5 years old. Australians using Information Australia were able to access resources relating to their heritage easily through the portal, and the usage figures indicate that these resources were very relevant to their information needs.

The subject areas of material requested ranged far and wide, including education, community history, family history, communications, law and geography. The greatest amount of requests for loans from the National Library, 80%, was for material in the subject areas of geography, history, literature and the social sciences. This contrasts significantly with the findings of a study of documents supplied from the National Library undertaken in 2003 which found that over 80% of requests were for science, health and medical resources.

The outcomes of the public library pilot are primarily:

  • a portal was developed which enabled delivery of federated searching through a “Google” type interface with support for “Getting”, the interface development provided lessons which were then incorporated into the new Kinetica search interface, Libraries Australia, released on 8 th December 2004;
  • the value to the Australian community of access to the nations collections through the Australian National Bibliographic Database was clearly demonstrated; and
  • access to “get” material in the National Library’s collection demonstrated a previously unmet need for access to Australian resources in a very wide range of subject areas and historical materials.

Conclusion

Australians have complex information needs. The results of the public library portal pilot suggests that library collections are vital to meetings these needs, but that lack of easy access to the nations collections and online resources has inhibited the use of libraries and our collections. The software developed for the portal, which has formed the basis of the new Kinetica search interface, Libraries Australia, is a very successful means of delivering access to the Australian National Bibliographic Database and Australian online resources to Australians.

The project combined Australian information resources, building on collaborative purchasing and the growing electronic collections to support access for all Australians. While discussion in library literature has focused on “find” and “get” services based on resource discovery and copy access, this project represents a major step towards full “access” combining direct access to electronic resources and the ability to request material not in digital form. The requesting, or “getting” component of the service has been demonstrated to be of great important to Australians. The new Libraries Australia interface enables libraries to have enhanced requesting options made available to their users, offering the ability to order through their library and in addition to order through a bookshop or document supply service.

The pilot will conclude in 2005, and has demonstrated that the partnership with of the National and public libraries and use of technology enables a seamless service for Australians linking print and electronic resources offers the opportunity to extend library services across the nation. Information Australia is an important step towards true access to Australian print and electronic resources for all Australians.

Information Australia has been replaced by the new Kinetica search interface, Libraries Australia. This interface combines developments in an improved interface with a “Google” style simple search, integration to provide “getting” and the ability to search full text databases such as APAFT.

Three major issues however remain to be resolved, funding models for community access for all Australians, support for resources to provide increased interlending in public libraries and authentication issues.

The most significant barrier to further development is the lack of a funding model for free public access to the National Bibliographic Database and Australian online full text resources. This need has been recognised in the Senate Environment, Communications, Information Technology and the Arts Inquiry into the Role of Libraries in the Online Environment which recommended:

The Committee recommends that the National Library of Australia receive additional funding to provide improved access to Kinetica for all Australian libraries and end users.

The Committee recommends

  • that the National Library of Australia identify a number of key databases for which national site licencing might be desirable; and
  • that additional government funding be extended to the National Library of Australia for this purpose.

While the Government response did not immediately identify funding for these activities discussions between all library sectors have occurred at the Site Licencing Forum and Peak Bodies meeting held at the National Library in December to investigate possible actions to achieve greater access to these resources.

For Australian libraries to continue to provide relevant and timely access to their collections Information Australia is an innovative demonstration of an approach that delivers a successful solution. The model was based on a reciprocal gratis interlending community, which through collaboration increased access to resources to support a wide range of personal, professional and community research. Collaborative activities have the potential to further increase the support libraries can provide the Australian community.

References

Australian Bureau of Statistics (2001) Public Libraries, Australia, 1999-2000 (8561.0) http://www.abs.gov.au/Ausstats/abs%40.nsf/b06660592430724fca2568b5007b8619/
08cdeae368a2a931ca256a780001d4db!OpenDocument

Australia. Parliament. Senate. Environment, Communications, Information Technology and the Arts References Committee (2003) Libraries in the online environment. Canberra: The Committee, 2003

Leyland, P. (2003) “Public libraries: community gateways in the information age”, National Library news, May 2003, http://www.nla.gov.au/pub/nlanews/2003/may03/article5.html

Moloney, K, Missingham, R. and MacKenzie, C. (2004) “A portal for the people: National Library & your local public library = new access paradigm”, paper presented to the VALA Conference held in Melbourne 2 - 6 February 2004, http://www.nla.gov.au/nla/staffpaper/2004/missingham1.html

National Library of Australia (2002) Direction for 2003-2005,http://www.nla.gov.au/library/directions.html

Back to the top