National Library of Australia - Home page
 

 

 

File formats

A file format is a 'particular way to encode information for storage in a computer file.' 1 A file extension generally indicates the format of a particular file. There are many types of file formats and many of them are proprietary. However, a growing awareness of the issues and difficulties related to proprietary formats especially within a digital preservation context has encouraged the development of open source (non-proprietary) file formats.

1. Wikipedia

Articles | Books | Events | Organisations and Websites | Online tutorial | Policies, Strategies & Guidelines | Projects and Case Studies | Bibliographies | Glossaries | Journals & Newsletters | Discussion Lists | Surveys | Tools | Evaluations

Articles

AONS II: Continuing the Trend Towards Preservation Software 'Nirvana'
Pearson, David (Date Created: Oct 2007) (Australia)
This paper presented at IPRES 2007, reports on the AONS II project undertaken by the National Library of Australia in conjunction with the Australian Partnership for Sustainable Repositories.The AONS software tool allows users to automatically monitor the status of file formats in their repositories and receive notification of obsolescence risks.
http://ipres.las.ac.cn/pdf/David%20Pearson-AONS%20II_%20continuing%20t
he%20trend%20towards%20preservation%20software%20'Nirvana'%20-%20.pdf
Screenshots of presentation and paper available from the the IPRES2007 website. http://ipres.las.ac.cn/
AONS II: Continuing the Trend Towards Preservation Software 'Nirvana'
Pearson, David In: iPRES2007 - Special issue (Date Created: 25 Jan 2008) (Australia)
This paper was originally presented at iPRES 2007 in Beijing China. It reports on the AONS II project undertaken by the National Library of Australia in conjunction with the Australian Partnership for Sustainable Repositories. The AONS software tool allows users to automatically monitor the status of file formats in their repositories and receive notification of obsolescence risks.
http://www.infotech.ac.cn/CN/article/downloadArticleFile.do?attachType
=PDF&id=510
New Technology for Library and Information Service, iPres2007 Special Issue, Vol1, 25 Jan 2008.
AONS System Documentation
Curtis, Joseph (Date Created: 29 Sep 2006) (Australia)
This is a technical document providing system documentation for AONS (Automatic Obsolescence Notification System). AONS is a software system that provides automatic notification of digital objects in a repository that may become obsolescent by using preservation information about the file formats. The document covers the architecture, registry and repository interface, AONS core and database structure.
http://www.apsr.edu.au/publications/aons_report.pdf
Safekept
Archival Data Format Requirements
Christensen, Steen S. (Date Created: Jul 2004) (Denmark)
This document discusses the requirements of the Netarkivet in terms of both file formats and metadata. The document outlines the basic requirements for storage formats that can be used for long term preservation. Different storage formats are then evaluated in relation to Netarkivet specifications and a recommendation for a suitable long term storage format is made.
http://netarkivet.dk/publikationer/Archival_format_requirements-2004.p
df
Safekept
Historical Archiving and Preserving PDF Files
Ockerbloom, John Mark In: RLG DigiNews (Date Created: Feb 2001)
This article examines the long-term preservation of Adobe Portable Document Format (PDF) files. Possible migration strategies for PDF documents are outlined and appropriate techniques for archiving are discussed.
http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070513/viewer/file5
58.html#feature2
Safekept
Historical Attributes of File Formats for Long Term Preservation of Scientific and Engineering Data in Digital Libraries
Barkstrom, Bruce R.; Folk, Mike (Date Created: 2002)
This paper was produced as part of the Investigating Scientific Data Management Applications with Electronic Records Project sponsored by NARA (National Archives and Records Administration). The paper discusses the characteristics of file formats that can be advantageous in the preservation and storage of scentific and engineering data. File formats are discussed in the context of archival storage and access, usability, data integrity and maintainability and durability.
http://www.ncsa.uiuc.edu/NARA/Sci_Formats_and_Archiving.doc
Automatic Format Identification Using Pronom and DROID. Digital Preservation Technical Paper No. 1
Brown, Adrian (Date Created: Oct 2005) (United Kingdom)
This technical paper describes DROID (Digital Record Object Identification), an open source software tool that performs automated identification of file formats.
http://www.nationalarchives.gov.uk/aboutapps/fileformat/pdf/automatic_
format_identification.pdf
New CODA Curation of Digital Assets 2007. Version 2: English translation of parts from the final report (2009)
LDB-Centrum (Date Created: 02 Apr 2009) (Sweden)
This document (20 p.) is an English version of one chapter plus an appendix (3) from the 2007 report in Swedish. It reports on the evaluation of the ability of four programs to identify the format of a file and version. The programs tested were: DROID, JHOVE, TriID and File Identifier. In the tests DROID and JHOVE scored best. The full report in Swedish (113 p.) which reports on both this project and another CODA_POP is available from the CODA-FORM website.
https://ldb.project.ltu.se/main.php/ldb.project.ltu.se/main.php/projec
ts/portalproject/docs/Publikationer/Svenska%20publikationer/CODA-2007_
English.pdf?fileitem=7475395
Corporate Alzheimer's : Coping With Forgotten File Formats
Waters, John K. (Date Created: 04 Apr 2006) (United States of America)
This brief article describes the latest developments in XML based 'open file' formats that claim to free documents from the restrictions of proprietary software for long term access.
http://www.law.com/jsp/PubArticle.jsp?id=900005450719
Defining File Format Obsolescence : A Risky Journey
Pearson, David; Webb, Colin In: International Journal of Digital Curation (Date Created: Jul 2008) (Australia)
This paper describes the process of defining file format obsolescence at the National Library of Australia for the AONS II project which was undertaken in conjunction with the Australian Partnership for Sustainable Repositories. File format obsolescence is a major risk for digital repositories. The project refined a software tool to help managers decide if preservation action is required.
http://www.ijdc.net/ijdc/article/view/76/78
Historical On DigiPaper and the Dissemination of Electronic Documents
Huttenlocher, Dan; Moll, Angela In: D-Lib magazine (Date Created: Jan 2000)
This article describes DigiPaper, which enables documents to be disseminated with the formatting fixed in place, as intended by the creator. It is a new image-based document representation which combines the compression technologies of Mixed Raster Content (MRC) color image model and token-compression to achieve much smaller files than current image formats.
http://www.dlib.org/dlib/january00/moll/01moll.html
In D-Lib Magazine, Vol 6, No. 1
Historical Digital Formats for Content Reproductions
Fleischhauer, Carl; Library of Congress (Date Created: 13 Jul 1998) (United States of America)
This document was prepared, in part, to offer guidance to applicants in the Library of Congress/Ameritech competition. Guidance for digitisation projects is offered under the following headings: Pictorial Materials, Textual Materials Reproduced as Searchable Text and Images, Textual Materials Reproduced as Images, Maps, Sound Recordings, Moving-image Materials, and Headers for Computer Files.
http://memory.loc.gov/ammem/formats.html
Digital Preservation : Will Your Files Stand the Test of Time?
Anderson, Cokie (Date Created: 2005)
This article canvasses various file formats and storage media and gives an oveview of their sustainability and vulnerabilities. The article proposes that preservation planning is the key to ensuring digital longevity.
http://iris.emeraldinsight.com/vl=7964756/cl=55/nw=1/rpsv/cgi-bin/link
er?ini=emerald&reqidx=/cw/mcb/07419058/v22n6/s3/p9
Also available in pdf. Library Hi Tech News incorporating Online and CD Notes, Vol. 22, no. 6 (2005) and by subscription online at Emerald Fulltext - http://iris.emeraldinsight.com
Evaluating File Formats for Long Term Preservation
Rog, Judith; van Wijk, Caroline (Date Created: Feb 2008) (Netherlands)
This paper outlines the Koninklijke Biliotheek's recently developed file format risk assessment methodology for long term preservation. The paper discusses whether it is possible to obtain universal agreement on a methodology particularly with the added challenges of web archiving. The authors seek feedback from other institutions.
http://www.kb.nl/hrd/dd/dd_links_en_publicaties/publicaties/KB_file_fo
rmat_evaluation_method_27022008.pdf
Historical File Formats for Preservation : Final Report
ERPANET (Electronic Resource Preservation and Access Network) (Date Created: May 2004) (Austria)
This is a report of the activities of the File Formats for Preservation Seminar at the Austrian National Library, 10-11 May 2004. Papers included information on a general and introductory level as well as papers dealing with specific areas including text, image and audiovisual formats.
http://www.erpanet.org/events/2004/vienna/Vienna_Report.pdf
New Gap Analysis: A Survey of PA Tool Provision
Houtman, Frank; van Bussel, Sara (Date Created: 12 Oct 2009)
This report (26 p.) prepared for the Planets project discusses the findings of a survey of file formats archived by cultural and scientific organisations; case studies of three specialised formats (DAISY, FITS, and sheet music formats); and considerations for determining gaps in the availability of preservation action (PA) tools. The report recommends that gap analysis needs to be undertaken every 2-3 years. It notes that individual organisations may have differing perceptions on whether a gap exits, based on their individual circumstances. Appendixes include: List of found file formats; and, List of migration tools.
http://planets-project.eu/docs/reports/PA2D3gapanalysis.pdf
Deliverable PA2/D3
General Study 11 Final Report : Selecting Digital File Formats for Long-Term Preservation
McLellan, Evelyn Peters (Date Created: Mar 2007)
This report prepared for the Domain 3 project unit of the InterPARES 2 Project analyses the current issues and trends in the selection of file formats for preserving digital records in the e-government, sciences and arts sectors. The report reviews documentation on the web sites of 20 repositories and 4 multi-national collaborative groups.
http://www.interpares.org/display_file.cfm?doc=ip2_file_formats(comple
te).pdf
Also available in French http://www.interpares.org/display_file.cfm?doc= ip2_file_formats_fichiers_numériques.pdf
Safekept
Historical Handling File Formats
Clausen, Lars. R. (Date Created: May 2004) (Denmark)
This report describes a part of “Internetbevaringsprojektet” (the Internet preservation project), which is a joint project between Statsbiblioteket (The State Library) and Det Kongelige Bibliotek (The Royal Library) in Denmark. The aim of the project is to archive the Danish part of the Internet. As part of this project, this report specifically deals with strategies in handling file formats to ensure long term preservation and accessibility. Some of the main ways to preserve access to information stored as digital objects are discussed, including capture, sequential conversion to new formats, conversion on demand, emulation and hardware preservation. The report concludes with a suggested strategy for dealing with file formats in an archival system, file preservation workflows and general recommendations for handling file formats.
http://netarchive.dk/publikationer/FileFormats-2004.pdf
JHOVE2 : a Next-Generation Architecture for Format-Aware Digital Object Preservation Formatting
Harvard University Library (Last Updated: 01 Jun 2006) (United States of America)
The paper discusses the scope of the JHOVE2 (JSTOR/Harvard Object Validation Environment) project proposal. This project will seek to retain existing JHOVE functionality and refactor the existing architecture, support enhancements, increase the range of supported formats, and develop modules for supporting key preservation processes. Input from the JHOVE user community will be sought as part of the design phase. The proposal was submitted by Harvard University, Portico, and Stanford University to the Library of Congress under the National Digital Information Infrastructure Preservation Program (NDIIPP) initiative.
http://hul.harvard.edu/jhove/JHOVE2-proposal.doc
Logic-Based Approach to the Formal Specification of Data Formats
Botchak, Arsene; Hartle, Michael; Mühlhäuser, Max; Schumann, Daniel (Date Created: 30 Sep 2008)
This paper given at iPRES 2008 (British Library Conference Centre, 29-30 September 2009) presents a logic based approach for the formal specification of data formats as a possibly infinite set of data format instances building upon the Bitstream Segment Graph model. To illustrate the approach a small subset of Portable Network Graphics (PNG) image format is used and described using a relatively small number of rules.
http://www.bl.uk/ipres2008/presentations_day2/46_Hartle.pdf
Historical Lossy or Lossless? File Compression Strategies Discussion at ALA
Dale, Robin In: RLG DigiNews (Date Created: 15 Feb 1999)
This 'technical feature' summarises discussions which took place at the American Library Association midwinter meeting in Philadelphia about lossy versus lossless file compression techniques. Six practitioners: Carl Fleischhauer, Louis Sharpe, III, Howard Besser, Peter Hirtle, Joy Paulson and Steven Puglia give their opinions on factors which affect the decision to use different compression techniques.
http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070513/viewer/file1
380.html#technical1
Historical PDF-A : a New Digital Preservation Format
LeFurgy, William G. (Date Created: May 2003) (United States of America)
Since 2002, a committee of government, business, and academic representatives has been exploring an approach to long-term preservation of text-based digital documents and working to establish an archival standard for Adobe’s Portable Document Format (PDF). Known as PDF-A, the potential standard intends to specify a limited, stable subset of PDF that must remain valid for a number of years. PDF-A is designed to meet three basic requirements: the needs of document producers, document users and archivists, while excluding features that hinder archival viability. The committee is sponsored by the Association for Information and Image Management (AIIM) and The Association for Suppliers of Printing, Publishing and Converting Technologies (NPES).
http://www.archivists.org/saagroups/gov/newsletters/May2003.asp#digita
l
in Government Record News : the newsletter of the Government Records Section of the Society of American Archivists, May/June 2003
Preservation of TeX/LaTeX Documents
Barnes, Ian (Date Created: 21 Jul 2006)
TeX is a mathematical typesetting programme language and LaTeX is based on TeX. The report addresses the suitability of TeX/LaTeX as a preservation format. The author maintains that in the long term these documents should be converted to an XML format such as Doc Book XML with embedded MathML.
http://www.apsr.edu.au/publications/LaTeX-preservation.pdf
Safekept
Historical Risk Management of Digital Information : A File Format Investigation
Arms, Caroline R. In: RLG DigiNews (Date Created: 15 Jun 2000) (United States of America)
This article is a review of the CLIR sponsored Cornell University Library study of the same name. The study concentrated on migration as a digital preservation strategy. There were seen to be three different categories of risk related to the infrastructure, file format and conversion process. Different methods and tools were applied to assess risk included workbooks, reader software and test migrations. It was found that risk could be quantified and access to the original file format specifications was crucial to evaluating risks.
http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070513/viewer/file1
579.html#technical1
Historical Role of format in Digital Preservation
Abrams, Stephen L. (Date Created: 2004)
The author argues that without a thorough understanding of the formats in which digital objects are encoded, long term preservation will be difficult. Within this context, the GDFR (Global Digital Format Registry) and JHOVE (JSTOR/Harvard Object Validation Environment), projects are discussed as examples of services and systems providing support to long term preservation. The development of a GDFR where formats will be available from a central source is explored. JHOVE, a project to automate the identification and validation of file formats, is also discussed.
http://oberon.emeraldinsight.com/10.1108/03055720410530997
Also available in pdf version and in Vine. Digital Preservation and Libraries. Part Two. Vol. 34, no. 2, 2004. pp. 49-55. ISSN 0305-5728
Significance is in the Eye of the Stakeholder
Dappert, Angela; Farquar, Adam (Date Created: Jun 2009)
This paper prepared for ECDL 2009 examines the concept of significance in digital preservation and presents a new model that places significance in the hands of stakeholders.
http://www.planets-project.eu/docs/papers/Dappert_Significant_Characte
ristics_ECDL2009.pdf
Significant Properties of Vector Images
Oxford Brookes University; System Simulation Ltd (Date Created: 2008) (United Kingdom)
This 74 p. report looks at Vector Images to identify their most important characteristics (Significant Properties) in the context of their long-term preservation and future re-use. It is one of four studies funded by JISC to investigate Significant Properties of Digital Objects.
http://www.jisc.ac.uk/media/documents/programmes/preservation/vector_i
mages.pdf
Safekept
Historical Survey and Assessment of Sources of Information on File Formats and Software Documentation : Final Report
Representation and Rendering Project, University of Leeds (Date Created: 2003) (United Kingdom)
This report was published by the UK Joint Information Systems Committee (JISC) as part of their Digital Preservation and Records Management programme. The report canvasses issues of rendering, migration and obsolescence; identifies sources of information for file format and software documentation; and lists a series of recommendations to combat software and hardware obsolescence.
http://www.jisc.ac.uk/uploaded_documents/FileFormatsreport.pdf
Tomorrow's File Endings : On Archiving Principles and Archiving Formats
Uneson, Marcus (Date Created: 2005) (Sweden)
This paper discusses general digital archiving principles, such using open standards and text based formats, particularly in regard to the archiving of scientific and scholarly publications. The efficacy of using various file formats in an archival setting are also discussed. These formats include pdf, pdf-a, tex/latex and xml.
http://www.sciecom.org/ojs/index.php/sciecominfo/article/viewFile/31/3
2
Towards a Curation and Preservation Architecture for CAD Engineering Models
Ball, Alexander; Ding, Lian; Patel, Manjula (Date Created: 29 Sep 2008)
This paper given at iPRES 2008 (British Library Conference Centre, 29-30 Sept. 2008) discusses preservation requirements of CAD models and presents two proof of concept tools developed to aid in their curation and preservation. RRoRIfe - Registry/Repository of Representation Information for Engineering - uses Representation Information, as defined by the OAIS Reference Model to advise on suitable migration strategies for migrating CAD models to archival or exchange formats. It is a simple preservation planning tool incorporating a registry of format characteristics and a registry of migration software. The second tool LiMMA -Lightweight Models with Multilayered Annotations - is an architecture for layering non-geometric information on top of a geometric model. It is a series of individual tools based around a common XML schema and workflow.
http://www.bl.uk/ipres2008/presentations_day1/17_Ball.pdf
Use of File Description Languages for File Format Identification and Validation
Dunckley, Matthew; Rankin, Stephen (Date Created: 2007) (United Kingdom)
This paper given at PV 2007 (October 9-11 2007, Munich) discusses the importance of automated file format identification and validation. It also presents and discusses a range of tools and generic software APIs that could solve the problems of file identification and structure validation.
http://www.pv2007.dlr.de/Papers/Dunckley_fileFormatIdentification.pdf
What? So what?: The Next-Generation JHOVE2 Architecture for Format-Aware Characterization
Abrams, Stephen; Cramer, Tom; Morrissey, Sheila (Date Created: 30 Sep 2008) (United States of America)
This paper was presented at IPRES 2008 (29-30 September, London) and provides an overview of the JHOVE2 project which is a two year project being funded by Library of Congress under the NDIIPP Program, the California Digital Library, Portico and Stanford University. The JHOVE2 project will develop and deploy a next-generation architecture providing enhanced performance, streamlined APIs, and significant new features.
http://confluence.ucop.edu/pages/viewpage.action?pageId=3932229
Paper and presentation slides available.
Windows Desktop Product Life Cycle Support and Availability Policies for Businesses
Microsoft corporation (Regularly Updated) (United States of America)
An article outlining Microsoft's policy for ongoing support for its desktop business products with timeliness and details on specific products.
http://www.microsoft.com/windows/lifecycle/default.mspx

Top

Books

Safekept
Historical Risk Management of Digital Information : A File Format Investigation
Gregory W. Lawrence, William R. Kehoe, Oya Y. Reiger, William H. Walters and Kenney R. Anne (Date Created: Jun 2000) (United States of America) Reviewed by: Risk Management of Digital Information : A File Format Investigation
This report is based on investigations by Cornell University Library with support from CLIR on the identification of risks to digital file formats during migration. A work book for risk assessment was developed, identifying risk categories, measuring risk using worksheets, matrixes and evaluation of conversion comparison software. Two migration case studies are also discussed: one for image files and one for numeric files.
ISBN: 1-887334-78-5
http://www.clir.org/pubs/abstract/pub93abst.html
Available on CLIR's website or paper copy for US $20.00

Top

Events

File Formats for Preservation
10 - 11 May 2004 , Austrian National Library, Wien
ERPANET (Electronic Resource Preservation and Access Network)
(Austria)
Erpanet seminar which discussed the requirements and best practices for archival file formats and issues such as file format registries and format validation. Includes links to presentations, evaluation reports, and further information from the Vienna Seminar.
http://www.erpanet.org/events/2004/vienna/index.php
JHOVE2 2009 Fall Workshop
07 Oct 2009 , Mission Bay Conference Center, San Francisco
JHOVE2 Project
(United States of America)
This free public half-day workshop to be be held in conjunction with iPRES 2009 is organised by the JHOVE2 Project Team. Presentations include: An overview of the JHOVE2 project; The role of characterization in digital curation and preservation workflows; Demonstration of the prototype; Third party development of modules; Architectural review; and integration into existing or planned systems and services.
http://jhove2.eventbrite.com/
nestor Seminar: The PDF/A File Format and Its Possible Applications in Digital Archives : Das Dateiformat PDF/A und seine Verwendungsmöglichkeiten in digitalen Archiven
, Bundesarchiv Koblenz NESTOR (Germany)
In this seminar organised by nestor, experts from the PDF/A Competence Center review and address issues surrounding PDF/A: 1. Characteristics of PDF/A as a suitable archive format; 2. Suitability of objects for archiving in PDF/A; 3. Creation of PDF/A files and conversion workflows; 4.Checking the validity of PDF/A files in archives ; 5. The limits of PDF/A; 6. PDF/A and metadata. Presentations in German available see seminar 4 18 November 2008.
http://nestor.sub.uni-goettingen.de/seminar_bundesarchiv/index.php

Top

Organisations and Websites

File Formats Blog
McGath, Gary (Regularly Updated)
Gary McGath is a software Engineer at Harvard University Libraries. His blog features 'news and comments about technical issues relating to file formats, file validation, and archival software.'
http://fileformats.blogspot.com/
FILExt - The File Extension Source
Computer Knowledge (Regularly Updated)
This web site provides a searchable database of file extensions and the programs that use them.
http://filext.com/

Top

Policies, Strategies & Guidelines

Digital Lifecycles And File Types : Final Report
Bates, Melanie; Loddington, Steve; Manuel, Sue; Oppenheim, Charles (Date Created: May 2006)
As part of the JISC funded Rights And Rewards In Blended Institutional Repositories Project, this report (60 p.) provides an overview of characteristics, advantages and disadvantages of various types of image and text file formats. It also includes information about managing the lifecycle of a digital item as well as standards and specifications concerning metadata (e.g. Dublin Core and METS). It discusses issues to consider when packaging content (e.g. for loading into a learning management system) and provides an overview of various content packaging tools. It concludes with an introduction to reference models and frameworks used to describe processes managed by repositories and the systems in which they operate.
http://hdl.handle.net/2134/1793
Digital Preservation Guidance Note 1 : Selecting File Formats for Long-Term Preservation. Issue 2.
Brown, Adrian (Date Created: Aug 2008) (United Kingdom)
This guidance note issued by The National Archives (TNA) Digital Preservation Department sets out a range of criteria t to help data creators and archivists make informed choices about file format issues and selecting file formats for use.
http://www.nationalarchives.gov.uk/documents/selecting-file-formats.pd
f
DPGN-01 Available in PDF and RTF version from http://www.nationalarchives.gov.uk/preservation/advice/digital.htm
Historical Digital Preservation Guidance Note 1: Selecting File Formats for Long-Term Preservation
Brown, Adrian (Date Created: 19 Jun 2003) (United Kingdom)
Produced by the Digital Preservation Department of the National Archives (UK) as part of its series of guidance notes on issues relating to the preservation of electronic records, this document focuses on the selection of file formats for long-term preservation. It describes various criteria that need to be considered when dealing with file format issues and selection. Superseded by Version 2 August 2008.
http://www.nationalarchives.gov.uk/documents/selecting_file_formats.pd
f
This document is the first in a series of guidance notes developed by the Digital Preservation Department, National Archives, UK. It is also available as an RTF document.
Digital Preservation Guidance Note 4 : Graphics File Formats. Issue 2
Brown, Adrian (Date Created: Aug 2008) (United Kingdom)
This guidance note issued by The National Archives (TNA) Digital Preservation Department provides advice on the general issues that should be considered by the creators and managers of electronic records when selecting file formats for use with images. Types of graphic format considered include raster graphics, vector graphics and metafiles.
http://www.nationalarchives.gov.uk/documents/graphic-file-formats.pdf
DPGN-04. Available in PDF and RTF from http://www.nationalarchives.gov.uk/preservation/advice/digital.htm
Historical Digital Preservation Guidance Note 4: Graphics File Formats
Brown, Adrian (Date Created: 09 Jul 2003) (United Kingdom)
Produced by the Digital Preservation Department of the National Archives (UK) as part of its series of guidance notes on the preservation of electronic records, this document focuses on selecting file formats for use with images. It provides summary information about the most common graphics file formats in current use to help users to make informed decisions regarding the selection of most suitable files. The guidelines provided are limited to static images. Superseded by version 2 August 2008.
http://www.nationalarchives.gov.uk/documents/graphic_file_formats.pdf
This document is the fourth in a series of guidance notes developed by the Digital Preservation Department, National Archives, UK. It is also available as an RTF document.
Framework for the Definition of Significant Properties
Knight, Gareth (Last Updated: 17 Mar 2008) (United Kingdom)
This document (49 p.) prepared for the InSPECT project outlines a methodology for the identification and description of significant properties contained by a digital resource. Digital object types considered are structured text, email, digital audio and raster images. It also provides a generic template which may be useful for the description of a wide range of digital resources.
http://www.significantproperties.org.uk/documents/wp33-propertiesrepor
t-v1.pdf
Safekept
Historical Handling File Formats
Clausen, Lars. R. (Date Created: May 2004) (Denmark)
This report describes a part of “Internetbevaringsprojektet” (the Internet preservation project), which is a joint project between Statsbiblioteket (The State Library) and Det Kongelige Bibliotek (The Royal Library) in Denmark. The aim of the project is to archive the Danish part of the Internet. As part of this project, this report specifically deals with strategies in handling file formats to ensure long term preservation and accessibility. Some of the main ways to preserve access to information stored as digital objects are discussed, including capture, sequential conversion to new formats, conversion on demand, emulation and hardware preservation. The report concludes with a suggested strategy for dealing with file formats in an archival system, file preservation workflows and general recommendations for handling file formats.
http://netarchive.dk/publikationer/FileFormats-2004.pdf
ISO 28500:2009 - Information and Documentation - WARC File format
International Organization for Standardization (Date Created: 13 May 2009)
This standard specifies the WARC file format which is an extension of the ARC format which has been used since 1996 to store files harvested on the web. The WARC format enables the recording of HTTP request headers, the recording of arbitrary metadata, the allocation of an identifier to every contained file, the management of duplicates and migrated records and the segmentation of the records. WARC files are intended to store every type of digital content, either retrieved by HTTP or another protocol.
http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm
?csnumber=44717
Draft version available from the BnF at http://bibnum.bnf.fr/WARC/index.html
PDF Guidelines : Recommendations for the Creation of PDF Files for Long-term Preservation and Access
Rog, Judith (Last Updated: 31 May 2007) (Netherlands)
This document provides guidelines on the standards that pdf files deposited with the e-Depot should comply with. The guidelines cover accessibility and structure, fonts, colour, compression, images and executable actions.
http://www.kb.nl/hrd/dd/dd_links_en_publicaties/PDF_Guidelines.pdf
Preservation & Storage Formats for Repositories
Repositories Support Project (Date Created: May 2008) (United Kingdom)
This Repositories Support Project briefing paper provides introductory advice on the preservation aspects of formats used for deposit and storage in repositories.
http://www.rsp.ac.uk/pubs/briefingpapers-docs/technical-preservformats
.pdf
RSP Briefing Paper
Sustainability of Digital Formats : Planning for Library Of Congress Collections : Still Images
Library of Congress (Regularly Updated) (United States of America)
The site operates as a technical information source on digital file formats. The information is intended to aid decision-making on options for long-term preservation of digital content. The site makes recommendations and gives reasons why some formats are preferable to others in terms of their capacity to support long-term access to content. This is a comprehensive information source produced by an organisation committed to maintaining and updating information about preservation of digital content.
http://www.digitalpreservation.gov/formats/content/still.shtml

Top

Projects and Case Studies

Alternative File Formats for Storing Master Images
Gillesse, Robert; Rog, Judith; Verheusen, Astrid (Date Created: 7 Mar 2008) (Netherlands)
This in depth report (64 pages) describes a project undertaken by the Koninklijke Bibliotheek on possible alternative file formats for storing master images from mass digitisation projects. The KB currently stores uncompressed TIFFs. This report considers JPEG 2000, PNG, JPEG and TIFF LZW as alternatives. It describes each format and considers the consequences for required storage capacity, image quality, long term sustainability and functionality.
http://www.kb.nl/hrd/dd/dd_links_en_publicaties/publicaties/Alternativ
e%20File%20Formats%20for%20Storing%20Masters%202%201.pdf
Digital Lifecycles And File Types : Final Report
Bates, Melanie; Loddington, Steve; Manuel, Sue; Oppenheim, Charles (Date Created: May 2006)
As part of the JISC funded Rights And Rewards In Blended Institutional Repositories Project, this report (60 p.) provides an overview of characteristics, advantages and disadvantages of various types of image and text file formats. It also includes information about managing the lifecycle of a digital item as well as standards and specifications concerning metadata (e.g. Dublin Core and METS). It discusses issues to consider when packaging content (e.g. for loading into a learning management system) and provides an overview of various content packaging tools. It concludes with an introduction to reference models and frameworks used to describe processes managed by repositories and the systems in which they operate.
http://hdl.handle.net/2134/1793
File Information Tool Set (FITS)
Harvard University Library (Date Created: Aug 2009) (United States of America)
The File Information Tool Set (FITS) has been developed by the Harvard University Library. FITS wraps several third-party open source tools, normalizes and consolidates their output, and reports any errors. The wrapped tools identify, validate, and extract technical metadata for various file formats. Tools currently wrapped include JHOVE, Exiftool, National Library of New Zealand Metadata Extractor, DROID, Ffident and File Utitlity. Two original FITS tools included are FileInfo and XmlMetadata. Other tools are being evaluated for inclusion.
http://code.google.com/p/fits/
JHOVE2
University of California Office of the President (UCOP) (Regularly Updated) (United States of America)
This wiki is the public information site for the JHOVE2 Project which builds on the JHOVE project. The JHOVE2 Project aims to refactor the existing JHOVE architecture and API, to support enhancements to existing JHOVE functionality and to develop JHOVE2 modules supporting important preservation-related processes. JHOVE2 is a 2 year project funded by the Library of Congress under the NDIIPP program, the California Digital Library, Portico and Stanford University. The wiki includes background to the project, use cases, project bibliography, details of mailing lists and links to related resources.
http://confluence.ucop.edu/display/JHOVE2Info/Home
JHOVE2 : a Next-Generation Architecture for Format-Aware Digital Object Preservation Formatting
Harvard University Library (Last Updated: 01 Jun 2006) (United States of America)
The paper discusses the scope of the JHOVE2 (JSTOR/Harvard Object Validation Environment) project proposal. This project will seek to retain existing JHOVE functionality and refactor the existing architecture, support enhancements, increase the range of supported formats, and develop modules for supporting key preservation processes. Input from the JHOVE user community will be sought as part of the design phase. The proposal was submitted by Harvard University, Portico, and Stanford University to the Library of Congress under the National Digital Information Infrastructure Preservation Program (NDIIPP) initiative.
http://hul.harvard.edu/jhove/JHOVE2-proposal.doc
Safekept
Historical Risk Management of Digital Information : A File Format Investigation
Gregory W. Lawrence, William R. Kehoe, Oya Y. Reiger, William H. Walters and Kenney R. Anne (Date Created: Jun 2000) (United States of America) Reviewed by: Risk Management of Digital Information : A File Format Investigation
This report is based on investigations by Cornell University Library with support from CLIR on the identification of risks to digital file formats during migration. A work book for risk assessment was developed, identifying risk categories, measuring risk using worksheets, matrixes and evaluation of conversion comparison software. Two migration case studies are also discussed: one for image files and one for numeric files.
ISBN: 1-887334-78-5
http://www.clir.org/pubs/abstract/pub93abst.html
Available on CLIR's website or paper copy for US $20.00
Significant Properties of Vector Images
Oxford Brookes University; System Simulation Ltd (Date Created: 2008) (United Kingdom)
This 74 p. report looks at Vector Images to identify their most important characteristics (Significant Properties) in the context of their long-term preservation and future re-use. It is one of four studies funded by JISC to investigate Significant Properties of Digital Objects.
http://www.jisc.ac.uk/media/documents/programmes/preservation/vector_i
mages.pdf

Top

Tools

AONS II: Continuing the Trend Towards Preservation Software 'Nirvana'
Pearson, David (Date Created: Oct 2007) (Australia)
This paper presented at IPRES 2007, reports on the AONS II project undertaken by the National Library of Australia in conjunction with the Australian Partnership for Sustainable Repositories.The AONS software tool allows users to automatically monitor the status of file formats in their repositories and receive notification of obsolescence risks.
http://ipres.las.ac.cn/pdf/David%20Pearson-AONS%20II_%20continuing%20t
he%20trend%20towards%20preservation%20software%20'Nirvana'%20-%20.pdf
Screenshots of presentation and paper available from the the IPRES2007 website. http://ipres.las.ac.cn/
File Information Tool Set (FITS)
Harvard University Library (Date Created: Aug 2009) (United States of America)
The File Information Tool Set (FITS) has been developed by the Harvard University Library. FITS wraps several third-party open source tools, normalizes and consolidates their output, and reports any errors. The wrapped tools identify, validate, and extract technical metadata for various file formats. Tools currently wrapped include JHOVE, Exiftool, National Library of New Zealand Metadata Extractor, DROID, Ffident and File Utitlity. Two original FITS tools included are FileInfo and XmlMetadata. Other tools are being evaluated for inclusion.
http://code.google.com/p/fits/

Top

Evaluations

Preserving the Data Explosion: Using PDF
Fanning, Betsy A. (Date Created: Apr 2008) (United Kingdom; United States of America)
This 27p. report reviews PDF and the newly introduced PDF/Archive (PDF/A) format as a potential solution for long-term digital preservation. It considers the use of PDF/A as part of a comprehensive records management program, to archive electronic documents as a standard to aid preservation and future retrieval.
http://www.dpconline.org/docs/reports/dpctw08-02.pdf
DPC Technology Watch Report 08-02

Top


Related Topics: Format registries | Format standards

Broader Topic: Standards

Home | About | Search | Browse topics | Feedback | Contributions | What's New?
NATIONAL LIBRARY OF AUSTRALIA,
Canberra, ACT 2600, AUSTRALIA
Telephone + 61 2 6262 1111; Facsimile +61 2 6257 1703;
Telephone Typewriter Number: 1800 026 372