National Library of Australia - Home page
 

 

 

Format registries

Detailed knowledge about data formats is required for many of the activities associated with digital preservation and digital repositories. In the past, this knowledge has tended to be gathered in a piecemeal fashion, for example, software companies have registries for their own proprietary software. In addition, the current MIME types registry maintained by the Internet Assigned Numbers Authority, suffers from a lack of granularity and insufficient standardised information on the formats.

Recently there have been trends towards more universally available and complete registries. The GDFR and PRONOM are examples of this type of development. In addition, prototypes such as FRED (Format REgistry Demonstration), produced by the University of Pennsylvania, function as a testbed to focus on the developmental and practical issues associated with format registries.

Examples of how registries can be used include identification (what is this object?), validation (is the object true to specification?), characterisation (what are its features?), risk assessment (is it at risk of obsolescence?) and delivery (how can this object be rendered?).

Articles | Books | Events | Organisations and Websites | Online tutorial | Policies, Strategies & Guidelines | Projects and Case Studies | Bibliographies | Glossaries | Journals & Newsletters | Discussion Lists | Surveys | Tools | Evaluations

Articles

Automatic Format Identification Using Pronom and DROID. Digital Preservation Technical Paper No. 1
Brown, Adrian (Date Created: Oct 2005) (United Kingdom)
This technical paper describes DROID (Digital Record Object Identification), an open source software tool that performs automated identification of file formats.
http://www.nationalarchives.gov.uk/aboutapps/fileformat/pdf/automatic_
format_identification.pdf
Safekept
Automating Preservation : New Developments in the PRONOM Service
Brown, Adrian In: Research Libraries Group (RLG) (Date Created: 15 Apr 2005) (United Kingdom)
PRONOM 4 is due to be launched in June 2005. This article discusses the revisions and enhancements available in the new version, including technical registry enhancements, automatic file format identification and the development of PRONOM unique identifiers (PUI).
http://www.rlg.org/en/page.php?Page_ID=20571#article2
Building a Robust Knowledge Base for Digital Formats
Ockerbloom, John Mark (Date Created: 21 Apr 2004) (United States of America)
FRED (Format REgistry Demonstrator) is a proof-of-concept prototype format registry which is part of the Mellon-funded TOM (Typed Object Model) project. This is a brief overview of FRED and includes characteristics of the registry, how formats are described and future challenges.
http://www.google.com/search?q=cache:MS2-0fFViSkJ:www.diglib.org/forum
s/spring2004/powerpoint/ockerbloom0404.ppt+Building+a+Robust+Knowledge
+Base+for+Digital+Formats&hl=en&ct=clnk&cd=1
Establishing a Global Digital Format Registry
Abrams, Stephen L. (Date Created: 2005)
This article beings by discussing format dependencies in digital repository operations, including ingest, migration, emulation and Universal Virtual Computer (UVC) dependencies. The author then discusses the data model, the service model and the governance and business model of the Global Digital Format Registry (GDFR).
http://muse.jhu.edu/demo/library_trends/v054/54.1abrams.html
Library Trends, Vol. 54, no. 1 (2005)
PRONOM
National Archives (UK) (United Kingdom)
PRONOM is a database system that stores and provides information about file formats and the application software needed to open them. It stores detailed descriptions of software applications, including what file formats an application can read and write to, any dependencies that may exist in order to run the application
http://www.nationalarchives.gov.uk/pronom/
Safekept
Historical PRONOM : a Practical Online Compendium of File Formats
Darlington, Jeffrey In: RLG DigiNews (Date Created: 15 Oct 2003) (United Kingdom)
PRONOM is a database established by the National Archives (UK) that stores and provides information on file formats and the software applications needed to read them. This article provides an overview of the purpose, content development strategies and future development plans for PRONOM. The latest version, the web-enabled PRONOM 3, is also described.
http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070511/viewer/file1
925.html#feature2
Robust Technologies for Automated Ingestion and Long-term Preservation of Digital Information
JaJa, Joseph (Date Created: 2006)
Description from abstract - this is an overview of the DIGARCH project and report on a number of significant advances achieved thus far. In particular, we highlight our contributions to the development of a novel architecture for the Global Digital Format Registry, the design of a highly reliable and scalable deep archive, and the development of the underpinnings of a policy-driven management of preservation processes. Challenges and future plans are briefly outlined.
http://doi.acm.org/10.1145/1146598.1146674
ACM International Conference Proceeding Series; Vol. 151, Proceedings of the 2006 international conference on Digital government research. Available via subscription.
Safekept
Historical Towards a Global Digital Format Registry
Abrams, Stephen L; Seaman, David (Date Created: Aug 2003)
Based on a preliminary working group of members from the international library and archive sector, this paper traces the group’s efforts in investigating the establishment a new global format registry for the purposes of supporting the long-term preservation of digital materials. The authors claim that the digital format registry “will maintain persistent, unambiguous bindings between public identifiers for digital representation formats and the significant syntactic and semantic properties of those formats”. The paper covers areas such as the registry’s mission statement, data and service models, governance structure and business issues. The development of the registry service model is informed by the ANSI X3.285 Metamodel for the Management of Shareable Data, the OASIS ebXML Registry and IANA MIME media types
http://www.ifla.org/IV/ifla69/papers/128e-Abrams_Seaman.pdf
This paper was given at the World Library and Information Congress: 69th IFLA General Conference and Council, 1-9 August, 2003 in Berlin.
Towards format repositories for web archives
Christensen, Niels H. (Date Created: Aug 2004) (Denmark)
This paper was presented at the International Web Archiving Workshop in 2004 and describes the challenges of file formats for web archives.
http://netarkivet.dk/publikationer/FormatRepositories-2004.pdf

Top

Organisations and Websites

FILExt - The File Extension Source
Computer Knowledge (Regularly Updated)
This web site provides a searchable database of file extensions and the programs that use them.
http://filext.com/
Safekept
FRED : a Format Registry Demonstration
University of Pennsylvania Library (Regularly Updated) (United States of America)
FRED (Format REgistry Demonstrator) is a proof-of-concept prototype format registry demonstrator developed from the Mellon-funded TOM (Typed Object Model) project. It's purpose is to provide some idea of the requirements that developing a global registry would entail. It can be used to demonstate how file format information can be presented and how it can be used, maintained and managed. There are links to the data model, a list of registered formats, and a discussion list.
http://web.archive.org/web/*/http://tom.library.upenn.edu/fred/
Safekept
Global Digital Format Registry (GDFR)
Harvard University Library (Regularly Updated)
In development since 2002, the GDFR (Global Digital Format Registry) is an ongoing international initiative. The GDFR working group comprises national and academic research libraries and archives. The GDFR was developed as a mechanism to provide trustworthy, sustainable and authoratitive information about data formats. This website also has information on data and service models, use cases and the GDFR mission statement.
http://hul.harvard.edu/gdfr/
Safekept
Typed Object Model (TOM)
University of Pennsylvania Library (Date Created: 2004) (United States of America)
Developed by the University of Pennsylvania Library, TOM (Typed Object Model) is a data format management system that describes the structure and behavior of a wide variety of data formats and information services. In 2003 and 2004 there was a Mellon Foundation funded initiative to apply this model to support online learning and digital preservation. TOM can be used to document data formats and services and assist in data format migration and other conversions. TOM's core software is being released as open source. This website has links to TOM's data model, standards and documentation.
http://web.archive.org/web/*hh_/tom.library.upenn.edu/

Top

Projects and Case Studies

Safekept
FRED : a Format Registry Demonstration
University of Pennsylvania Library (Regularly Updated) (United States of America)
FRED (Format REgistry Demonstrator) is a proof-of-concept prototype format registry demonstrator developed from the Mellon-funded TOM (Typed Object Model) project. It's purpose is to provide some idea of the requirements that developing a global registry would entail. It can be used to demonstate how file format information can be presented and how it can be used, maintained and managed. There are links to the data model, a list of registered formats, and a discussion list.
http://web.archive.org/web/*/http://tom.library.upenn.edu/fred/
Safekept
Global Digital Format Registry (GDFR)
Harvard University Library (Regularly Updated)
In development since 2002, the GDFR (Global Digital Format Registry) is an ongoing international initiative. The GDFR working group comprises national and academic research libraries and archives. The GDFR was developed as a mechanism to provide trustworthy, sustainable and authoratitive information about data formats. This website also has information on data and service models, use cases and the GDFR mission statement.
http://hul.harvard.edu/gdfr/
Safekept
Typed Object Model (TOM)
University of Pennsylvania Library (Date Created: 2004) (United States of America)
Developed by the University of Pennsylvania Library, TOM (Typed Object Model) is a data format management system that describes the structure and behavior of a wide variety of data formats and information services. In 2003 and 2004 there was a Mellon Foundation funded initiative to apply this model to support online learning and digital preservation. TOM can be used to document data formats and services and assist in data format migration and other conversions. TOM's core software is being released as open source. This website has links to TOM's data model, standards and documentation.
http://web.archive.org/web/*hh_/tom.library.upenn.edu/

Top

Tools

DROID
National Archives (UK) (Regularly Updated) (United Kingdom)
Purpose : Developed by the National Archives (UK) DROID (Digital Record Object Identification) is an open source software tool that performs automated identification of file formats. These signatures are stored in an XML signature file, generated from information recorded in the PRONOM technical registry. New and updated signatures are regularly added to PRONOM, and DROID can be configured to automatically download updated signature files from the PRONOM website.
Documentation : Full documentation including XML schemas and download instructions are available at this URL.
http://droid.sourceforge.net/wiki/index.php/Introduction

Top


Related Topics: File formats | Format standards

Broader Topic: Standards

Home | About | Search | Browse topics | Feedback | Contributions | What's New?
NATIONAL LIBRARY OF AUSTRALIA,
Canberra, ACT 2600, AUSTRALIA
Telephone + 61 2 6262 1111; Facsimile +61 2 6257 1703;
Telephone Typewriter Number: 1800 026 372