About file conversion and loading

Once you have uploaded files via the Data Contribution Service, Libraries Australia staff convert and load files onto the ANBD.

File conversion

MARC conversion is performed on every file before it is loaded to the ANBD.

Most conversions are generic and are performed automatically on every file that is supplied to Libraries Australia.

In addition, there are specific conversions available for customer files based on profiles set up for each organisation. These conversions are done to change minor variations in the MARC formats supplied, and to ensure that all bibliographic and holdings records loaded to the ANBD meet data quality standards, internal data storage formats and RIS load program validation routines.

Successfully converted files are sent for loading. Any unsuccessfully converted records are placed in an error file for review by Libraries Australia staff or the contributing library.

File loading

Converted files are forwarded to the Data Contribution Service load program and are generally loaded in the order of priority assigned to them.

The highest priority is assigned to current files from Libraries Australia customers.

Lower priority is generally assigned to retrospective customer file loads, and large file loads (in excess of 20,000 records).

Libraries Australia reserves the right to determine priorities for file loading based on the significance of the data for national resource sharing.

During the file loading process, incoming records are either:

  • added to the database as new records

  • matched to an existing database record and merged with it

  • tentatively matched with database records and sent to a review file, or

  • rejected due to problems with the incoming data.

Tentative matches are resolved by adding them to the ANBD. Although records are very rarely rejected, occasionally there can be a problem in the format of a particular field that was not detected during the conversion process.

Error logs and file load reports are monitored and checked against previously kept statistics to see that the files are fully loaded and to check for any other signs of data problems.

You will receive email notification of completed file loads and availability of reject files. You can then view summary loading statistics online for files that have been loaded, and download files of rejected records, via Libraries Australia Administration.

Rejected records

Records rejected during loading are placed on the Libraries Australia FTP server for retrieval, correction and resubmission. They can also be downloaded from Libraries Australia Administration.

Reject files come in 2 formats:

  • errors.mrc: file of MARC records without the error messages

  • errors.txt: text file of records (in full view) with the error messages.

For a comprehensive explanation of error messages in the ‘errors.txt’ report, see:

If records contain both errors and warnings, they will only be reported in the files that contain errors—for example, ‘errors.mrc’ or ‘errors.txt’.

You will receive an email notification of these reject files. Rejected files are automatically deleted from the Libraries Australia FTP server after 90 days.

A separate email alert is generated and sent to your account when reject files are available and created.

You can also access the following reports:

  • review.txt: text file of records (in full view) that have tentatively matched database records and require further review

  • unprocessed.txt: text file of records (in full view) that could not be processed correctly through all matching and merging steps and were not added to the database or used for updating existing records

  • warnings.mrc: file of MARC records which generated warnings without the warning messages

  • warnings.txt: text file of records (in full view) which generated warnings with the warning messages (note that these warnings do not prevent records from processing through the Data Contribution Service or being matched and merged).

Detailed statistical reports are available in Libraries Australia Administration.