Quality Improvement Plan for the Australian National Bibliographic Database and CJK Database

An ongoing program of database maintenance and quality tasks is carried out by Libraries Australia Database Services staff. To assist in prioritising and managing these tasks, a Quality Improvement Plan has been developed. The following table lists a wide variety of tasks in priority order, which will improve quality of records and indexes in both the ANBD and CJK databases. The resolution of tasks listed below will improve the quality of data migrated to the new Libraries Australia service..

CURRENT

Task Strategy Task Priority Start Finish

Duplicate removal

Remove duplicate records reported by Libraries Australia users including serial duplicates.

Remove duplicate records identified in other ways.

Remove duplicate records reported by CJK users including serial duplicates.

Remove duplicate records identified in other ways.

.

About 400 duplicate records are manually removed each month including those reported by Libraries Australia users and those discovered during ANBD activities.

Identify and resolve duplicates from a generated list of most popular Australian works.

Nearly 30,000 duplicate records are detected and removed each month by the automated file loading facility.

In the CJK database nearly 2,000 duplicate records are detected and removed each month by the automated file loading facility.

1 Ongoing  

Identify and remove redundant data in duplicate or multiple fields (eg MARC 300 504).

Strategy may involve changes to Merge algorithm to prevent addition of multiple fields. The small numbers of these fields being generated are being fixed as found. Some changes have been made to BatchLink to exclude records with multiple fields from batch loads.

3 Ongoing  

Improve Match/Merge algorithm to minimise the incidence of duplicate records due to the lack of match points

Implement improvement to Match/Merge to improve the matching of records:

eg Amicus Number/Library of CongressNumber matching (implemented)
Title matching changes in test Pending:
Immutable number conversion validation
Expand list of 010 prefixes to be filtered out on load
Form of item matching improvements
Review ISBN/ISSN matching
Review merging behaviour - duplicated data
Review verification level/language - duplicate headings
1 Ongoing Ongoing

Identify and resolve multiple Library of Congress Control Numbers (6452 records):
Task 1: Same LC numbers appear twice in MARC 010 field.
Task 2: Different LC numbers appear in 010 (NR) field

Two separate lists for records with and without holdings have been generated for each problem. Task 1 requires analysis to determine which number to delete.
Task 2 is manual (in progress)

3

2
Ongoing Ongoing

Identify and fix incorrect filing indicators in Title field which cause indexing and retrieval problems

A Programmer assisted technique will identify affected records Activity scheduled for annual basis.

2 Ongoing  

Many records in the ANBD contain incorrect coding for Musical or Non Musical sound recordings

A report generated from the database will identify a high percentage and these can be corrected using an automated process

So far just over 18,000 records have been reviewed of which 6,000 required correction. Records targeted so far have been those related to the Music Australia database. But the scope is now widening to include a wider range, which may require manual correction.

2 15 October 03 3rd Quarter 2004

EACC and CCCII character codes in the CJK database will be mapped to Unicode

An automated process of mapping from EACC/CCCII to Unicode will be provided by the system vendor.

2 TBA 4th Quarter 2004

Maintain Subject heading where Library of Congress and Australian forms conflict (includes headings referred to Subject Headings Advisory Panel)

Strategy comprises:

  1. Aligning Australian headings with Library of Congress authority file
  2. Perpetual maintenance strategy for terms that can't be aligned
  3. Building in global mapping to next iteration of Libraries Australia
2 Ongoing  

Identify and fix incorrectly coded LCCSH and MeSH authorities (2797 records)

Initial investigations have commenced for LCCSH to determine if an automated fix is possible, or identify scope of manual fix.

3 Ongoing  

Ongoing authority file maintenance

  1. Identify and resolve incorrect headings reported by Libraries Australia users
  2. Identify and resolve incorrectly coded Australian Name Authorities -

Carry out Authority file maintenance on CJK:
Identify and resolve incorrect headings reported by CJK users

Identify through through subject categories such as music, science, history, and resolve using automated process to attach Australian codes. Use automated means where possible.

3 Ongoing  

Remove records which have not been modified in more than five years, and which do not contain Australian content codes or Table of Contents data, and have no holdings attached.
Ensure that records are available from other sources (eg RLIN, OCLC)

Records which are older than 5 years, which do not contain the Australian Content Indicator and which have not been edited, and which have no holdings attached and which are not TOC records are identified and deleted as surplus to requirements.

National Diet Library Records over 5 years old and without an ISSN and with no holdings have been deleted from the CJK database. Some 50,000 records have been deleted.

2 To end March 2004 1.6 million bibliographic records deleted. Ongoing

On CJK Implement new load and export tables with EACC/CCCII <-> Unicode conversion

Will enable greater sharing of data between CJK service and CJK members and better services for Australian libraries.

     

Identify and remove ASCII hex '1D' within the record. Character is used solely as the end of record marker and should never appear inside a record.

Improve records for retrieval by customers.

1 2 Jan 04 2 nd quarter 2004

Identify and fix remove/replace MARC tags and indicators.

Tags and indicators have been identified. A strategy for clean up is still to be agreed and implemented. Additional tags such as MARC 886 may need to be added to the existing list.

1 1 Dec 03 ongoing

Identify and remove Chinese, Japanese and Korean records from ANBD and load to CJK.

Identify and improve quality of linked authority records in the ANBD

Identify by language code and other means. Delete records which a re duplicated in the CJK database, but move holdings to the CJK database.

Copy records which are not currently in CJK database to the database.

Before deleting Chinese bibliographic records from the ANBD, identify linked authority records. Those attached to records other than CJK records will be converted to Pinyin romanisation if not already in Pinyin, and checked for quality.

Orphan headings will be removed.

1 17 Nov 03 End 4th quarter 2004

Continuing cleanup of new CJK additions to ANBD

Regular removal from ANBD and addition to CJK

  Ongoing  

CJK and ANBD consolidation

Related to task above

1 17 Nov 03 End 4th Quarter 2005

Removal of near duplicate name headings

A specification for removal of headings has been written, but needs to be refined. May also involve changes to MatchMerge algorithm to prevent the loading of near duplicate terms.

1 1 Feb 04 3 rd quarter 2004

Clean up records with duplicate classification numbers to improve the display of records.

Strategy yet to be determined.

Investigate changes to merge algorithm to prevent further occurrences.

3    

Correct Vietnamese records containing spacing circumflex instead of combining circumflex in double diacritic.

Search out and convert (Circumflex should really be diacritic closest to the letter but has often been input in wrong place)
Requires further investigation and specification of a fix.(Records need to be viewed in Libraries Australia to be able to see the problem)

3 3 Feb 04 4 Apr 04

Correct records not displaying correctly in Web or Client

Data migration to the redeveloped Libraries Australia database provides a chance to identify and fix these records.

2   30 Sept 04