A Guide to the collection assessment process
Assessing a library collection is one part of the larger
task of managing a library collection. In smaller libraries collection
management may be one of many tasks performed by the professional staff, and in
larger libraries collection management may be a significant part of the work of
several staff members.
No matter what the size of the library or its
collection, the objectives of collection management are the same -
responsiveness and accountability. Library collections are created to respond
to the actual and anticipated demands of the users of those collections.
Library collections are managed so that librarians can, in building the
collection, determine how well resources are being applied.
This is an updated version of the Chapter on assessment
in the Australian Collection Assessment Manual: A Collection Assessment
Guide, compiled by Margaret Henty, National Library of Australia,
1992.
A. Planning and preparation
-
- Involve decision makers early in the process to
establish goals, objectives and time tables for collection assessment. Clearly
define the objectives of the project and expected uses of any assessment
reports.
- Review the range of collection assessment
information available at the Australian Libraries Gateway site and make
yourself familiar with the structure for recording information about your
collection in the Gateway.
- Review the subject divisions and decide which
ones should be completed.
- Decide on the most appropriate assessment level,
eg Category or Subject level.
- Determine the appropriate assessment techniques.
A list of collection assessment tools and references is
provided.
- Review existing documentation, such as collection
development and/or selection policies, statistics, and any previous assessment
data. Check out the collection assessment reports and collection development
policies from other libraries listed in the Australian Libraries Gateway.
- Determine data to be gathered; generally the more
information gathered, the more complex and costly the process will be. Identify
exactly how information will be used or is necessary.
- Identify the staff, users, volunteers, etc who
will be involved in the process.
- Provide staff training and resources; and record
time required for all activities in order to project future requirements, and
to measure results of the assessment.
- Assign and schedule tasks; establish timetable
for completion, estimate costs (time for planning, conducting the study and
evaluating the results; supplies; and computer time if needed).
- B. Conducting the assessment
-
- Gather needed data and tools;
- Carry out the assessment (teamwork is
recommended) - keep records of time spent, methods used anddecisions reached;
- Carry out the assessment (teamwork is
recommended); keep records of time spent, methods used; decisions reached;
- Monitor progress and results;
- Determine collection strengths and weaknesses;
and
- Identify unique strengths, e.g. special
collections vi. Add comments, noting special characteristics, chronological or
geographic strengths or weaknesses.
- C. Review and submit the data to
the Australian Libraries Gateway
-
- Review the collection assessment information you
have collected with key staff.
- Enter the assessment information for your entry
in the Australian Libraries Gateway, using the subject collecting level update
screen.
- D. Use of the collection assessment
information
-
- Incorporate collection assessment results into
the library's collection development policy;
- Use data for budget planning; and
- Use collection assessment data in collection
management: deselection, preservation, etc.
Although rating a collection subject by subject is a
complex task, it need not be formidable. Trust in your own judgment and
experience with the collection. Some studies have shown that professional
judgment of collection quality is typically more accurate than inaccurate. This
is not to suggest that assessment verification is unimportant. It does mean
that judgments on collections can be made with confidence and checked by using
one or more techniques, as time permits.
The following criteria, adapted from materials prepared
by the Research Libraries Group in the United States, summarise factors
librarians should keep in mind as they examine a subject area of the
collection.
- Number of titles: Count of
shelflist, or approximation based on 30 volumes per metre of shelf occupancy.
- Checking the collection against standard
bibliographies reflects:
- Principal authors: Are the
standard, chief, or more important authorities and authors included?
- Principal works: Are the
classic, standard, essential and important works in the collection?
- Primary Sources: Are the
critically edited original texts and documents included? How extensively?
- Criticism/commentary/interpretation: How complete is
secondary monographic or critical treatment?
- Chronological coverage: Are older
and newer materials consistently represented? Should they be?
- Complete sets: Are sets and series
well represented in the collection? Are they complete?
- Periodical coverage: How extensive
is periodical coverage of the subject? Are runs complete or broken? Are the
chief titles included?
- Access to periodicals: Are the major
indexes or abstracts in the field available either in paper or on-line?
- Other formats and/or special
collections: Is the collection significantly strengthened by
electronic documents, audio-visual materials, microfilms or other special
collections?
- Language: Is the collection
primarily in English or does it include extensive foreign language materials?
There are many different techniques for evaluating and
assessing collections, but they can all be considered as either
collection-centred or client-centred.
Client-centred techniques measure how
the collection is used by library users. Examples of these techniques are
circulation studies, interlibrary loan statistics, shelf availability studies,
and various user studies.
Collection-centred techniques examine
the content and characteristics of the collection to determine the size, scope,
and/or depth of a collection, often in comparison to an external standard.
Examples of these techniques include checking lists, counting holdings, and
expert evaluation.
There are excellent descriptions of assessment
techniques in the literature. One of the most useful is the American Library
Associations Guide to Evaluation of Library Collections, 1989. Five
techniques for establishing Existing Collection Strength levels are
discussed on the following pages:
- Shelflist measurement (p. 22)
- Shelf scanning (p. 24)
- List checking (p. 25)
- Evaluation by outside expert (p. 26)
- Citation analysis (p. 27)
These measures are a combination of quantitative and
qualitative techniques. The measures are not completely objective; indeed,
there is a high degree of subjectivity. However, the experience of collection
evaluation librarians has been that the techniques:
- Are applicable to all types of libraries;
- Provide reliable data;
- Should be used in combination as a means of
complementing and verifying; and
- Confirm the ability of librarians to apply their
knowledge, expertise, and judgement.
The following provides an overview of each of these
assessment techniques, giving their advantages, disadvantages and procedures
for applying each assessment technique.
This method yields quantitative information on the
number of titles, percentage of total collection, median age, and language of
the collection. Shelf lists no longer exist in all libraries in card form.
Where possible, use an electronic equivalent to derive the same information.
Advantages:
- Easy to do;
- Provides objective, quantifiable data;
- Provides data for comparison with other libraries;
and
- Can be carried out by support staff.
Disadvantages:
- Reflects catalogued holdings only;
- Does not measure quality;
- Provides a gross measure; and
- Data may not be accurate, especially if used for
comparison, because of variations in sampling and local procedures for
cataloguing and shelflisting.
Procedure
With a shelflist card file, the following procedures can
be used:
- Count the number of cards in one centimetre; use
several samples and average.
- Using a ruler or other reliable measure, measure the
total number of centimetres in the shelflist.
- Multiply the number of cards in one centimetre by the
total number of centimetres in the shelflist to arrive at the approximate
number of titles held.
- An alternative to steps 1, 2 and 3 is to measure the
length of the shelflist and multiply that number by 40 (a standard figure for
the number of cards per centimetre) to obtain the total number of titles.
- Count the total number of cards in the Division being
assessed. Record '# of Holdings' and '% of Holdings' and Language. To determine
the median age of the group of titles, tally the copyright dates
chronologically, either by decade or by five year period. Then determine the
point at which one half of the dates fall below and one half fall above.
The tally sheet for a group of 19 titles might look like
this:
pre-1970 11111 1111 9
1970-1979 111 3
1980-1989 11111 5
1990+ 11 2
Sampling
There are occasions when the quantity of titles to be
examined is so large as to discourage individual handling eg to determine the
median age of a set of books, or to determine language codes. In such cases, it
is easier to select a sample and work on that portion only.
The figure provides information on sample sizes for
different quantities of titles.
Confidence Level: 95% Reliability: +/- 5%
% in Population Assumed to be 50%
Population Sample Populations Sample Population Sample
Size Size Size Size Size Size
10 9 230 144 1400 310
15 14 240 147 1500 305
20 19 250 151 1600 309
25 23 260 155 1700 313
30 27 270 158 1800 316
35 32 280 162 1900 319
40 36 290 165 2000 322
45 40 300 168 2200 327
50 44 320 174 2400 331
55 48 340 180 2600 334
60 52 360 186 2800 337
65 55 380 191 3000 340
70 59 400 196 3500 346
75 62 420 200 4000 350
80 66 440 205 4500 354
85 69 460 209 5000 356
90 73 480 213 6000 361
95 76 500 217 7000 364
100 79 550 226 8000 366
110 85 600 234 9000 368
120 91 650 241 10000 369
130 97 700 248 15000 374
140 102 750 254 20000 376
150 108 800 259 30000 379
160 113 850 264 40000 380
170 118 900 269 50000 381
180 122 950 273 60000 381
190 127 1000 277 70000 382
200 131 1100 284 120000 382
210 136 1200 291 160000 383
220 140 1300 296 1000000 383
This technique employs physical examination of the
materials on the shelf in relation to the 'Criteria for Assessing a
Collection'.
Advantages:
- Provides immediate, relevant results;
- Can be done quickly;
- Provides overall view of the size, scope and quality
of the collection;
- Builds on knowledge and expertise of the assessors;
- Assessor sees what the users see; and
- Can be used to accomplish goals other than
assessment.
Disadvantages:
- Subjective and impressionistic;
- Depends on knowledge and expertise of assessor;
- Results may be biased if conducted by the librarian
who developed that collection or currently selects for it; and
- Some materials may not be on the shelf.
Procedure
- Select classification area. Work with partner if
possible.
- Gather necessary tools.
- Determine scope: examine every item or a sample.
- Determine location of other materials in the
collection that need to be considered: elecetronic documents, periodicals,
vertical files, commercial databaes, reference tools, etc.
- Look for such things as: physical condition of the
materials (weeding needed?)/types of materials/language/serial runs: complete?
Broken? Bound? With or without deficiencies?/scope, extent of the
collection/special problems/multiple copies.
- Record findings.
This technique measures holdings against authoritative
lists of what has been published.
A list of bibliographies and indexes, arranged by
subject, is available for your for subject list
checking.
Advantages:
- A wide variety of lists is available;
- Provides concrete, objective picture of holdings;
- Verifies selection decisions;
- Quantitative results useful for budget justification,
accreditation studies, etc.;
- May be carried out by support staff;
- Results easily summarised and suitable for comparison
with other collections; and
- Provides specific information for purchasing to
strengthen collection.
Disadvantages:
- Lists become outdated quickly;
- May be difficult to evaluate their validity,
usefulness, relevance, etc.
Procedure
- Decide on lists appropriate to the subject and goals
of the library see the subject specific guides listed in its URL;
- Decide whether to check the lists completely or by
sampling;
- Assign staff responsibility and check lists against
card catalogue or automated system;
- Record the number of titles held that are listed in
the bibliography being checked; and
- Determine percentage of library holdings in relation
to number of titles on each list.
- Analyse results and integrate findings with results
of other techniques to determine the collection level.
This technique involved an external subject specialist
in some aspect of the assessment process, working with library staff. For
example a faculty member, librarian, or other professional, or a public or
special library user with specialised knowledge surveys a portion of the
collection.
Advantages:
- Produces a fresh look at the collection;
- Facilitates mutually advantageous communications
between librarians and others such as administration, academic staff and
users;
- Librarians gain knowledge of research
interests;
- Users gain knowledge of
collection;
- The library's public is involved with
the library.
Disadvantages:
- Judgment may be impaired by bias, narrow or
specialised view, or lack of understanding of the library's collection policy;
- Subjective; and
- Outside experts may be difficult to locate or gain
commitment from.
Procedure
- Tailor the experience to fit the library's
needs.
- Make certain that there is clear understanding of the
objectives, conditions, and scope of the expert's involvement.
- Assign library staff liaison and establish
appropriate check points if experts are working independently.
- Avoid self-styled experts.
- Acknowledge expert's contribution.
- Integrate findings with results of other techniques
to determine the collection level.
NOTE: This method is easily combined with shelf
scanning.
This technique is more applicable to current materials
and research collections in university or special libraries, or for analysis of
textbooks used in a particular school curriculum. It involves compiling a list
of citations from footnotes and bibliographies of scholarly books and articles.
In essence, librarians create a specialised list from the cited references.
Advantages:
- Lists can be created to meet specific needs;
- Useful if published lists are not available;
- Works well for interdisciplinary areas or specialised
fields;
- Timely and current;
- Responsive and impressive to academic staff, users of
special libraries, etc.;
- Sensitive to latest developments;
- Citations could be computer generated;
- Frequently used to develop core lists of primary
journals; and
- Can help identify candidates for cancellation or
storage.
Disadvantages:
- Time consuming and labor intensive;
- Not appropriate for broad fields;
- Unknown variables govern original citations such as:
credentials of author, availability of information resources to the author and
selection of source materials, and bias of author. A citation is not an
inherent guarantee of quality;
- Tends to under-represent materials important for
format, context or background but which are infrequently cited;
- Susceptible to popular trends; and
- Citations do not follow consistent bibliographic
standards.
Procedure
- Sources of citations include dissertations, theses,
scholarly and specialised books and articles, special reports, works by best
authors in the field, reference tools, and electronic databases;
- Determine the sampling method;
- Compile list;
- Check against library holdings;
- Tabulate and analyse results; and
- Integrate findings with results of other techniques
to determine the collection level.
For further information on ways of assessing your
collection look at Collection assessment: tools and
references.
Last modified: August 10 2004.