National Library of Australia


A Guide to the collection assessment process

Assessing a library collection is one part of the larger task of managing a library collection. In smaller libraries collection management may be one of many tasks performed by the professional staff, and in larger libraries collection management may be a significant part of the work of several staff members.

No matter what the size of the library or its collection, the objectives of collection management are the same - responsiveness and accountability. Library collections are created to respond to the actual and anticipated demands of the users of those collections. Library collections are managed so that librarians can, in building the collection, determine how well resources are being applied.

This is an updated version of the Chapter on assessment in the Australian Collection Assessment Manual: A Collection Assessment Guide, compiled by Margaret Henty, National Library of Australia, 1992.


1. Outline of the collection assessment process

A. Planning and preparation

  1. Involve decision makers early in the process to establish goals, objectives and time tables for collection assessment. Clearly define the objectives of the project and expected uses of any assessment reports.

  2. Review the range of collection assessment information available at the Australian Libraries Gateway site and make yourself familiar with the structure for recording information about your collection in the Gateway.

  3. Review the subject divisions and decide which ones should be completed.

  4. Decide on the most appropriate assessment level, eg Category or Subject level.

  5. Determine the appropriate assessment techniques.

  6. Review existing documentation, such as collection development and/or selection policies, statistics, and any previous assessment data. Check out the collection assessment reports and collection development policies from other libraries listed in the Australian Libraries Gateway.

  7. Determine data to be gathered; generally the more information gathered, the more complex and costly the process will be. Identify exactly how information will be used or is necessary.

  8. Identify the staff, users, volunteers, etc who will be involved in the process.

  9. Provide staff training and resources; and record time required for all activities in order to project future requirements, and to measure results of the assessment.

  10. Assign and schedule tasks; establish timetable for completion, estimate costs (time for planning, conducting the study and evaluating the results; supplies; and computer time if needed).

B. Conducting the assessment

  1. Gather needed data and tools;

  2. Carry out the assessment (teamwork is recommended) - keep records of time spent, methods used anddecisions reached;

  3. Carry out the assessment (teamwork is recommended); keep records of time spent, methods used; decisions reached;

  4. Monitor progress and results;

  5. Determine collection strengths and weaknesses; and

  6. Identify unique strengths, e.g. special collections vi. Add comments, noting special characteristics, chronological or geographic strengths or weaknesses.

C. Review and submit the data to the Australian Libraries Gateway

  1. Review the collection assessment information you have collected with key staff.

  2. Enter the assessment information for your entry in the Australian Libraries Gateway, using the subject collecting level update screen.

D. Use of the collection assessment information

  1. Incorporate collection assessment results into the library's collection development policy;

  2. Use data for budget planning; and

  3. Use collection assessment data in collection management: deselection, preservation, etc.

2. Criteria for assessing a collection

Although rating a collection subject by subject is a complex task, it need not be formidable. Trust in your own judgment and experience with the collection. Some studies have shown that professional judgment of collection quality is typically more accurate than inaccurate. This is not to suggest that assessment verification is unimportant. It does mean that judgments on collections can be made with confidence and checked by using one or more techniques, as time permits.

The following criteria, adapted from materials prepared by the Research Libraries Group in the United States, summarise factors librarians should keep in mind as they examine a subject area of the collection.

  1. Number of titles: Count of shelflist, or approximation based on 30 volumes per metre of shelf occupancy.

  2. Checking the collection against standard bibliographies reflects:

  3. Chronological coverage: Are older and newer materials consistently represented? Should they be?

  4. Complete sets: Are sets and series well represented in the collection? Are they complete?

  5. Periodical coverage: How extensive is periodical coverage of the subject? Are runs complete or broken? Are the chief titles included?

  6. Access to periodicals: Are the major indexes or abstracts in the field available either in paper or on-line?

  7. Other formats and/or special collections: Is the collection significantly strengthened by electronic documents, audio-visual materials, microfilms or other special collections?

  8. Language: Is the collection primarily in English or does it include extensive foreign language materials?

3. Assessment techniques

There are many different techniques for evaluating and assessing collections, but they can all be considered as either collection-centred or client-centred.

Client-centred techniques measure how the collection is used by library users. Examples of these techniques are circulation studies, interlibrary loan statistics, shelf availability studies, and various user studies.

Collection-centred techniques examine the content and characteristics of the collection to determine the size, scope, and/or depth of a collection, often in comparison to an external standard. Examples of these techniques include checking lists, counting holdings, and expert evaluation.

There are excellent descriptions of assessment techniques in the literature. One of the most useful is the American Library Associations Guide to Evaluation of Library Collections, 1989. Five techniques for establishing Existing Collection Strength levels are discussed on the following pages:

These measures are a combination of quantitative and qualitative techniques. The measures are not completely objective; indeed, there is a high degree of subjectivity. However, the experience of collection evaluation librarians has been that the techniques:

The following provides an overview of each of these assessment techniques, giving their advantages, disadvantages and procedures for applying each assessment technique.

3.1 Shelflist measurement

This method yields quantitative information on the number of titles, percentage of total collection, median age, and language of the collection. Shelf lists no longer exist in all libraries in card form. Where possible, use an electronic equivalent to derive the same information.

Advantages:

Disadvantages:

Procedure

With a shelflist card file, the following procedures can be used:

  1. Count the number of cards in one centimetre; use several samples and average.
  2. Using a ruler or other reliable measure, measure the total number of centimetres in the shelflist.
  3. Multiply the number of cards in one centimetre by the total number of centimetres in the shelflist to arrive at the approximate number of titles held.
  4. An alternative to steps 1, 2 and 3 is to measure the length of the shelflist and multiply that number by 40 (a standard figure for the number of cards per centimetre) to obtain the total number of titles.
  5. Count the total number of cards in the Division being assessed. Record '# of Holdings' and '% of Holdings' and Language. To determine the median age of the group of titles, tally the copyright dates chronologically, either by decade or by five year period. Then determine the point at which one half of the dates fall below and one half fall above.

The tally sheet for a group of 19 titles might look like this:


               	pre-1970     	11111 1111	9
               	1970-1979	111		3
		1980-1989	11111		5
		1990+		11		2	

Sampling

There are occasions when the quantity of titles to be examined is so large as to discourage individual handling eg to determine the median age of a set of books, or to determine language codes. In such cases, it is easier to select a sample and work on that portion only.

The figure provides information on sample sizes for different quantities of titles.

	Confidence Level: 95%	Reliability: +/- 5%
	% in Population Assumed to be 50%

	Population  Sample  Populations  Sample  Population  Sample
	Size        Size    Size         Size    Size        Size

            10           9      230         144     1400        310 
            15          14      240         147     1500        305
            20          19      250         151     1600        309
            25          23      260         155     1700        313
            30          27      270         158     1800        316
            35          32      280         162     1900        319
            40          36      290         165     2000        322
            45          40      300         168     2200        327
            50          44      320         174     2400        331
            55          48      340         180     2600        334
            60          52      360         186     2800        337
            65          55      380         191     3000        340 
            70          59      400         196     3500        346  
            75          62      420         200     4000        350
            80          66      440         205     4500        354 
            85          69      460         209     5000        356
            90          73      480         213     6000        361
            95          76      500         217     7000        364   
           100          79      550         226     8000        366
           110          85      600         234     9000        368
           120          91      650         241    10000        369 
           130          97      700         248    15000        374
           140         102      750         254    20000        376 
           150         108      800         259    30000        379 
           160         113      850         264    40000        380
           170         118      900         269    50000        381
           180         122      950         273    60000        381
           190         127     1000         277    70000        382
           200         131     1100         284   120000        382 
           210         136     1200         291   160000        383
           220         140     1300         296  1000000        383 

 

3.2 Shelf scanning

This technique employs physical examination of the materials on the shelf in relation to the 'Criteria for Assessing a Collection'.

Advantages:

Disadvantages:

Procedure

  1. Select classification area. Work with partner if possible.
  2. Gather necessary tools.
  3. Determine scope: examine every item or a sample.
  4. Determine location of other materials in the collection that need to be considered: elecetronic documents, periodicals, vertical files, commercial databaes, reference tools, etc.
  5. Look for such things as: physical condition of the materials (weeding needed?)/types of materials/language/serial runs: complete? Broken? Bound? With or without deficiencies?/scope, extent of the collection/special problems/multiple copies.
  6. Record findings.

3.3 List checking

This technique measures holdings against authoritative lists of what has been published.

Advantages:

Disadvantages:

Procedure

  1. Decide on lists appropriate to the subject and goals of the library see the subject specific guides listed in its URL;
  2. Decide whether to check the lists completely or by sampling;
  3. Assign staff responsibility and check lists against card catalogue or automated system;
  4. Record the number of titles held that are listed in the bibliography being checked; and
  5. Determine percentage of library holdings in relation to number of titles on each list.
  6. Analyse results and integrate findings with results of other techniques to determine the collection level.

3.4 Evaluation by outside expert

This technique involved an external subject specialist in some aspect of the assessment process, working with library staff. For example a faculty member, librarian, or other professional, or a public or special library user with specialised knowledge surveys a portion of the collection.

Advantages:

Disadvantages:

Procedure

  1. Tailor the experience to fit the library's needs.
  2. Make certain that there is clear understanding of the objectives, conditions, and scope of the expert's involvement.
  3. Assign library staff liaison and establish appropriate check points if experts are working independently.
  4. Avoid self-styled experts.
  5. Acknowledge expert's contribution.
  6. Integrate findings with results of other techniques to determine the collection level.

NOTE: This method is easily combined with shelf scanning.

3.5 Citation analysis

This technique is more applicable to current materials and research collections in university or special libraries, or for analysis of textbooks used in a particular school curriculum. It involves compiling a list of citations from footnotes and bibliographies of scholarly books and articles. In essence, librarians create a specialised list from the cited references.

Advantages:

  1. Lists can be created to meet specific needs;
  2. Useful if published lists are not available;
  3. Works well for interdisciplinary areas or specialised fields;
  4. Timely and current;
  5. Responsive and impressive to academic staff, users of special libraries, etc.;
  6. Sensitive to latest developments;
  7. Citations could be computer generated;
  8. Frequently used to develop core lists of primary journals; and
  9. Can help identify candidates for cancellation or storage.

Disadvantages:

Procedure

  1. Sources of citations include dissertations, theses, scholarly and specialised books and articles, special reports, works by best authors in the field, reference tools, and electronic databases;
  2. Determine the sampling method;
  3. Compile list;
  4. Check against library holdings;
  5. Tabulate and analyse results; and
  6. Integrate findings with results of other techniques to determine the collection level.

Last modified: September 05 2012.