Digital Preservation Matters

Geospatial Data Preservation.

2011-11-15T23:27:00.001-07:00

Geospatial Data Preservation. Website. November 2011.
The Geospatial Resource Center is being developed as a finding tool for freely available web-based resources about the preservation of geospatial information. A variety of selected resources are being added, including reports, presentations, standards, and information about tools for preparing geospatial assets for long-term access and use. The resources are indexed to enable searching of titles and are categorized to facilitate discovery by choosing among topics, resource types, or both. The website contains many valuable resources. A few resources from these three categories:

Education & Training

Appraisal and selection of geospatial data for preservation
Best Practices for Geospatial Programs
Copyright Quickguide

Tools & Software

Cost Estimation Toolkit
Conversion tools for geospatial data
Geospatial metadata tools

Policies & Benefits

Collection policies,
Content standards,
Policies on Open geospatial data access and preservation

OAIS / TDR presentation at FDLP.

2011-10-23T21:38:00.003-06:00

OAIS / TDR presentation at FDLP. James A. Jacobs. Federal Depository Library Conference. Free Government Information. October 2011. [PDF]
A presentation giving an introduction to the "Reference Model for an Open Archival Information System" (OAIS) and the "Audit And Certification Of Trustworthy Digital Repositories" (TDR). This includes slides with speaker notes and a nice handout about related information with links. Every library decision should assess the impact of digital issues. Notes:

OAIS
1. It defines the functional concepts of a long-term archive with consistent, unambiguous terminology.
2. It gives us a functional framework for designing archives, and a functional model.
3. It gives us a standard for “conformance.”
4. It is a “Reference Model” that describes functions; it is not an “implementation”
5. Some key OAIS concepts are:
   - Designated Community: An identified group of potential Consumers who should
      be able to understand a particular set of information.
   - Description of roles and functions in the information life cycle.
   - The Long Term: Long enough for there to be concern about changing technologies,
      new media and data formats, and a changing user community.
   - Preserved content must be usable according to the designated community

TDR
Documents what is being done and how well it is being done.
Provides 109 “metrics” for measuring conformance to OAIS in three areas:
1. Organizational Infrastructure
2. Sustainability and succession plan
3. Digital Object Management
4. Technical Infrastructure And Security Risk Management

Cite Datasets and Link to Publications

2011-10-22T01:41:00.000-06:00

Cite Datasets and Link to Publications. Digital Curation Centre. 18 October 2011.
The DCC has published a guide to help authors / researchers create links between their academic publications and the underlying datasets. It is important for those reading the publication to be able to locate the dataset. This recognizes that data generated during research are just as valuable to the ongoing academic discourse as papers and monographs, and in many cases the data needs to be shared. "Ultimately, bibliographic links between datasets and papers are a necessary step if the culture of the scientific and research community as a whole is to shift towards data sharing, increasing the rapidity and transparency with which science advances."

This guide has identified a set of requirements for dataset citations and any services set up to support them. Citations must be able to uniquely identify the object cited, identify the whole dataset and subsets as well. The citation must be able to be used by people and software tools alike. There are a number of elements needed, but the "most important of these elements – the ones that should be present in any citation – are the author, the title and date, and the location. These give due credit, allow the reader to judge the relevance of the data, and permit access the data, respectively." A persistent url is needed, and there are several types that can be used.

Audit And Certification Of Trustworthy Digital Repositories.

2011-10-22T01:19:00.002-06:00

Audit And Certification Of Trustworthy Digital Repositories. CCSDS. September 2011. [PDF, 77pp.]

The Management Council of the Consultative Committee for Space Data Systems (CCSDS) has published this manual of recommended practices. It is based on the 2003 version from RLG. “The purpose of this document is to define a CCSDS Recommended Practice on which to base an audit and certification process for assessing the trustworthiness of digital repositories. The scope of application of this document is the entire range of digital repositories.”

The document addresses audit and certification criteria, organizational infrastructure, digital object management, and risk management. It is a standard for those who audit repositories; and, for those who are responsible for the repositories, it is an objective tool they can use to evaluate the trustworthiness of the repository.

National Archives Digitization Tools Now on GitHub

2011-10-20T23:14:00.000-06:00

National Archives Digitization Tools Now on GitHub. NARAtions. October 18, 2011.
The National Archives has begun to share applications developed in-house to facilitate digitization workflows. These applications have significantly increased the productivity and improved the accuracy and completeness of the digitization.Two digitization applications, “File Analyzer and Metadata Harvester” and “Video Frame Analyzer” are publicly available on GitHub.

File Analyzer and Metadata Harvester: This allows a user to analyze the contents of a file system or external drive and generate statistics about the contents, generate checksums, and verify that there is a one-to-one match of before and after files. The File Analyzer can import data in a spreadsheet, and can match and merge results with auxiliary data from an external spreadsheet or finding aid.
Video Frame Analyzer: This is used to objectively analyze technical properties of individual frames of a video file in order to detect quality issues within digitized video files. It reduced the time to do quality checks by 50%.

Digital Preservation-Friendly File Formats for Scanned Images.

2011-10-17T00:46:00.002-06:00

Digital Preservation-Friendly File Formats for Scanned Images. Bill LeFurgy. The Signal. October 12, 2011.

Some digital file formats are better for preservation than others. The best format for preservation is one where the content can be viewable accurately regardless of changes in hardware, software or other technical changes. The Library of Congress has created a web resource to help in selecting file formats, and which will help in understanding how effective formats for long-term preservation.

Disclosure of specifications and tools for validating technical integrity
Adoption by the primary creators and users of information resources
Openness to direct basic and non-propriety tools
Self-documentation of metadata needed to render the data as usable information or understand its context
Degree to which the format depends on specific hardware, operating system, or software for rendering the information and how difficult that may be.
Extent that licenses or patents may inhibit the ability to sustain content.
Technical protection mechanisms. Embedded capabilities to restrict use in order to protect the intellectual property.

Using these factors has helped determine formats that may be more sustainable than others.

To Save and Project Fest: Long Live Cinema!

2011-10-17T00:10:00.001-06:00

To Save and Project Fest: Long Live Cinema! J. Hoberman. The Village Voice. October 12, 2011.
Digital might be the future of the motion-picture medium, but for film preservation, it’s a mixed blessing. Archivists make it clear that digital technology is part of the solution—and part of the problem. Digital cinema is itself difficult to preserve, subtly distorts (by “improving”) the celluloid image, even as it often dictates (through commercial considerations) those movies deemed worthy of preservation. New York Times DVD critic Dave Kehr has pointed out that instead of increasing access, each new distribution platform (from 35mm to 16mm, VHS, DVD, Blu-ray, and online streaming) has narrowed the range of titles in active distribution and diminished the proportion of available films. Film restoration is also the restoration of cultural memory.

Tim O'Reilly - Keynote for 2011 NDIIPP/NDSA Partners Meeting.

2011-10-16T23:45:00.012-06:00

Tim O'Reilly - Keynote for 2011 NDIIPP/NDSA Partners Meeting. Tim O'Reilly. Library of Congress website. October 7, 2011.
This is a 31 minute video talking about digital preservation. The things that turn out to be historic often are not thought of being historic at the time. You can’t necessarily do preservation from the institution level. You have to teach the preservation mindset. Like Wikipedia; it is designed to keep all earlier versions. We should think about what kind of tools we need to build digital preservation into our everyday activities.

There will be a whole new dimension to digital. Imagine what will happen in situations if only digital books and maps are available and then they become unavailable. That world may be closer than we think. Imagine a world if there are no print books. What would you need to keep the digital materials available? It turns out that digital actually increases the manufacturing cost of books. We need to have tools with digital preservation designed in, not necessarily in the way we think of scholarly preservation, but in terms of increasing the likelihood that things will survive.

What should the web’s memory look like? There is an obligation to preserve the things that matter. We are engaged in the wholesale destruction of our history because we aren’t thinking about what we do as important to our descendants. Think of yourselves as people who are engaged in a task that matters to everyone. As we move into an increasingly digital world, preservation won’t be just the concern of specialists, but of everyone. One of the arguments for open source is simply to preserve the code. There have been a number of examples of technical companies not having their source code after they stop supporting it. Preserving everything may get in the way of our preserving the things that are important.

Innovation, Disruption and the Democratization of Digital Preservation

2011-10-13T22:52:00.006-06:00

Innovation, Disruption and the Democratization of Digital Preservation. Bill LeFurgy. Agogified. October 10, 2011.

Interesting article about innovation and society. It asks the question about digital preservation: Is innovation the key to dealing with all that valuable digital data? "When considered from the popular perspective of innovation, digital preservation looks like a straightforward challenge for libraries, archives, museums and other entities that long have kept information on behalf of society." But it isn't that easy, since technology changes much faster than society's conventions and institutions. "Innovation is not a safe, orderly or controllable process. It sends out big ripples of disruption with an unpredictable impact." Libraries are being bounced around because of such disruption and the traditional methods are not suited to address the changes. "All this means that the ability of traditional institutions to fully meet the need for digital preservation is in doubt."

But with these changes comes a change in the people playing a role in preserving digital materials. Some see a greater role for individuals in digital preservation. There is a great need for designing preservation functionality into tools used to create and distribute digital content to enable content creators to be involved in the digital stewardship. "Ultimately, we have to hope that innovation pushes along the trend toward the democratization of digital preservation. The more people who care about saving digital content, and the easier it is for them to save it, the more likely it is that bits will be preserved and kept available."

Abingdon firm gets Queen's seal of approval

2011-10-11T22:28:00.002-06:00

Abingdon firm gets Queen's seal of approval. Oxford Journal. 22 September 2011.
Tessella has been awarded one of the UK’s most prestigious business awards for their collaboration with a public sector organization in developing a unique system for preserving digital information. Their product, Safety Deposit box which came out in 2003, is now used by governments in seven countries.

Piggybacking to Avoid Going Down the Rabbit Hole, or What I Learned at the First DPOE Workshop.

2011-10-09T23:55:00.000-06:00

Piggybacking to Avoid Going Down the Rabbit Hole, or What I Learned at the First DPOE Workshop. Sam Meister. The Signal. October 7, 2011.
This is an excellent post about the Digital Preservation Outreach and Education (DPOE) initiative’s first ever Train the Trainer Baseline Workshop and the experience gained there. The workshop went over the core principles and concepts (Audience, Content, Instructors, Events) and the six modules that make up the curriculum (Identify, Select, Store, Protect, Manage, Provide). The group of 24 trainers broke into six regional groups to work through the modules and develop specific strategies to present the material of individual modules. "As the first group of trainers to review, analyze, revise and disseminate this curriculum, the result of a multi-year development process, we would be the “pioneers” for the DPOE program. To me, this made clear the role and level of responsibility that would be expected of us throughout the rest of the workshop and beyond."

New Guidelines: CrossRef DOIs to be Displayed as URLS.

2011-10-07T23:29:00.005-06:00

New Guidelines: CrossRef DOIs to be Displayed as URLS. Carol Anne Meyer. D-Lib Magazine. September/October 2011.
CrossRef, a not-for-profit association of more than 1000 scholarly publishers, revised its recommendations for CrossRef Digital Object Identifiers (DOIs) to specifies that DOIs on the web use a URL format. The previous practice of putting "doi:" in the ID is discontinued, and also that publishers create DOIs that are as short as possible.

From Link Rot to Web Sanctuary: Creating the Digital Educational Resource Archive (DERA).

2011-10-07T23:09:00.007-06:00

From Link Rot to Web Sanctuary: Creating the Digital Educational Resource Archive (DERA). Bernard M. Scaife. Ariadne. July 2011.
One of the tasks was to fix the broken links in the catalogue. A report showed that of about 16,000 links to external resources, about 1,200 were non-functional (7.5%). There were ways to fix many of these, but about 10% of the links referred to documents which no longer existed. Many of these were government publications. The question was how to do this differently. They looked at adding materials into their own repository, which would allow them to solve the link rot problem while "building in a core level of digital preservation and increasing the discoverability of these documents. We were convinced that a citation which linked to a record in a Web archive was far more likely to survive than one which did not."

They needed to clarify the intellectual property rights, add descriptive metadata, such as the type of document, a collection name, subjects, and the organization that created the document. They also found that they "had to accept all common file formats at present. In practice, the majority are pdf, some MS Word and a few Excel files. It would, for preservation purposes, be preferable to convert and ingest in PDF/A format, at least for the textual formats. However our view was that the small overhead of batch migrating to that format at a later stage means it would be better to spend time upfront now on metadata rather than file conversion. We felt that this was a pragmatic response which meant that we would be working within the spirit of digital preservation best practice." Also, they found that "data-based formats such as Excel cannot be meaningfully integrated into a full-text search and that these objects would benefit from better representations."

Other things they learned include

Placing files in a repository gives digital preservation to key documents in the subject field and eradicates the link rot problem.
Adding high-quality metadata enhances the resource and allows it to hold its head high and become an integral part of a library's collection.
A library can play an important role in preserving content as part of its long-term strategy and ensure high-quality resources remain available.
The added value of being able to search the full text provides a potentially very rich resource for researchers.

Future plans are to build up content levels and to integrate these resources with the regular library content,

More (digital) wake-up calls for academic libraries

2011-10-07T21:08:00.008-06:00

More (digital) wake-up calls for academic libraries. Rick Luce. LIBER 2011. Duurzame toegang blog. June 2011.
The topic was the core business of academic libraries: serving researchers and the scientific research process. There are many changes taking place in the sciences: "zetabytes of data; dynamic, complex data objects that require management; communities and data flows becoming much more important than static library collections, etc." The warning to academic libraries was that if libraries do not develop those services the new researcher needs, someone else will, and then there is no future for the research library. We need a "fundamental transformation process that will affect every aspect of the ‘library’ business." The library needs to provide a repository between the scientific process and IT infrastructure that supports and preserves workflows.

Graduates To Sow Seeds of New Training Program Across U.S.

2011-10-05T00:01:00.001-06:00

Graduates To Sow Seeds of New Training Program Across U.S. Bill LeFurgy. The Signal. October 3, 2011.
The inaugural class of digital preservation training workshop was held at the Library of Congress on September 20-23. The 24 professionals were selected from a nationwide pool. The DPOE Workshop was a workshop model designed to produce a national corps of trainers equipped to teach others basic principles and practices of preserving digital materials. “What’s unique about this workshop,” said George Coulbourne, Executive Program Manager, “is that we designed it for people who are going to be actual practitioners of digital preservation. This is not for administrators or managers, but for the novice practitioner. It’s also intended to be as open-source and low-cost as possible. We hope this event accelerates a new national movement in open, accessible digital-preservation training.”
Those who were trained will take back and present what they learned in their home regions, including by holding one or more training events in digital preservation by mid-2012.

ADS and the Data Seal of Approval – case study for the DCC.

2011-09-28T00:02:00.001-06:00

The ADS and the Data Seal of Approval – case study for the DCC. Jenny Mitcham and Catherine Hardman. Digital Curation Centre website. 2010.

This page describes the experience of Archaeology Data Service in applying for the Data Seal of Approval (DSA). It provides some practical information about the DSA application process and outlines issues the ADS faced in undertaking the process, and several potential benefits they see from the self-certification.

“When undertaking to curate data for the foreseeable future (and beyond) the concept of ‘trust’ is of paramount importance. Yet in a young discipline such as digital archiving, it is very difficult to demonstrate the potential for longevity of curation.”

The Assessment Manual can be downloaded from the DSA website, which includes details of the 16 guidelines, the minimum requirements, and some guidance notes. In the spirit of the openness the DSA recommends that the main policy and procedure documents should be accessible the world at large. One of the benefits mention is it shows to users and depositors that the archive has a set of standards is meeting them.

Fujitsu CTO: Flash is just a stopgap

2011-09-27T23:36:00.000-06:00

Fujitsu CTO: Flash is just a stopgap. Chris Mellor. The Register. 8 August 2011.
Flash memory according to the CTO is "beset with problems that will become unsolvable". The increases in flash density come at the expense of the ability to read and write data. "Each shrink in process geometry, from 3X to 2X and onto 1X, shortens flash's endurance", and brings its additional problems of access speed and endurance.

Tomorrow will be too late – (born) digital in library special collections.

2011-09-27T21:27:00.007-06:00

Tomorrow will be too late – (born) digital in library special collections. 28 June 2011.

Report from the annual conference of LIBER, the Association of European Research Libraries: It appears that according to recent studies, research libraries are still functioning within an analogue paradigm. Many libraries have digitized collections and provide online access, but their digitization efforts “mostly lack strategic planning, access is still mostly provided in a controlled way (for a limited group of users), preservation issues are still not being addressed adequately, and born-digital material (including audiovisual content) is blatently missing from collections and collection plans.” If libraries stay in their comfort zone of digital access in a controlled network, their information role will become insignificant. They need:

A knowledge of the user groups and their behaviours
Content, access, and preservation strategies
Strategic alliances
Permanent innovation
An open mind

There are opportunities for libraries but they need to transition their methods.

Google helps put Dead Sea Scrolls online

2011-09-27T21:13:00.000-06:00

Google helps put Dead Sea Scrolls online. BBC News. 26 September 2011.

Ultra-high resolution images of several Dead Sea Scrolls are now available on line. Five scrolls have been digitized (1,200 megapixel images); they are The Temple Scroll, The War Scroll, The Community Rule Scroll, The Great Isaiah Scroll, and The Commentary of Habakkuk Scroll.

Practical Approaches to Electronic Records: What Works Now (ppt)

2011-09-23T00:45:00.001-06:00

Practical Approaches to Electronic Records: What Works Now. Chris Prom, et al. August 30, 2011.
This is a PowerPoint of a presentation given by several people at SAA. A few notes from the slides:
Adjectives that are undesirable when describing an archive: Undercounted, undermanaged, inaccessible.
Basic requirements for the digital archive:

Perform a virus check
Capture descriptive metadata about the folders and files
Document the file formats
Record checksums for the files
Document the actions taken over time

Meeting the Challenge of Media Preservation: Strategies and Solutions.

2011-09-23T00:25:00.004-06:00

Meeting the Challenge of Media Preservation: Strategies and Solutions." Indiana University Bloomington. September 2011. (128 page pdf.)
This excellent study is the result of a year of research and planning to address the problems an earlier report in 2009. It looks at the preservation and conservation of audio, video, and film, including: guiding preservation principles, facility planning, prioritization, digitization methodologies, strategies for film, principles for access, technological infrastructure needs, and engagement with campus units and priorities. It is specifically for the university, but the information and recommendations are of interest to others. Their mission is to preserve the time-based media holdings of Indiana University so that they may be accessible. They estimate their media holdings at more than 560,000 audio, video, and film objects, and nearly all on obsolete formats. And they estimate they only have a fifteen- to twenty-year window of opportunity to digitally preserve audio and video holdings. They propose to collect rich descriptive and technical metadata to support digitization and future interpretation and management of digital content.

The media preservation crisis impacts every institution with media collections.
Because campus holdings are very large and time pressures great, even high-efficiency workflows may not preserve everything in time.
Not every recording is an appropriate candidate for long-term preservation.
Research, instruction, curation, and public availability are core university missions supported by media preservation efforts.
Access is the end goal of any preservation work, and it must be developed in tandem with
media preservation efforts.
Access to preserved holdings is critical to the success of the project and to the realization of its value to the campus.
The vision is an era characterized by a wealth of media content preserved long term and made accessible and integrated into campus research and instruction.
We live in a watershed moment in which acute challenges demand a coordinated effort to address dramatic technological and cultural changes in the way users access time-based media.
Target: high resolution audio preservation and production masters—24 bit, 96 kHz sample rate.

Some recommendations are that a 10,000-square-foot Indiana Media Preservation and Access Center is built employing 25 staff: administrators, audio and video engineers, film specialists, processing technicians, and IT support. The annual output is projected at 2-3PB of data per year with a total fifteen-year target of 39PB of data storage. The first year of work will be focused on developing solutions to the challenges posed by legacy media. The second year will begin developing management strategies and workflows for file-based born digital recordings.

Their guiding principles include: Curatorial Responsibility; Standards and Best Practices; Online Accessibility; Description Services; usability of metadata; copyright strategies; Access Digitizing and Preservation. Their efforts need to be combined into a trusted digital repository. "Preservation metadata requirements need to be defined, and tools need to be developed to support audio and video preservation package validation, technical metadata capture, and repository ingest."

This is a long study but is well worth the time to read all the way through it. This university link also includes other related materials, such as the media preservation survey, and a brochure "Our History is at Risk".

Library of Congress To Launch New Corps of Digital Preservation Trainers. Library of Congress.

2011-09-16T18:20:00.000-06:00

Library of Congress To Launch New Corps of Digital Preservation Trainers. Library of Congress. The Signal. September 16, 2011. Bill LeFurgy.
The Digital Preservation Outreach and Education program at the Library of Congress will hold its first national train-the-trainer workshop on September 20-23, 2011, in Washington, DC.

The DPOE Baseline Workshop will produce a corps of trainers who are equipped to teach others, in their home regions across the U.S., the basic principles and practices of preserving digital materials. Examples of such materials include websites; emails; digital photos, music, and videos; and official records. The intent of the workshop is to share high-quality training in digital preservation, based upon a standardized set of core principles, across the nation.

Long-term Preservation for Spatial Data Infrastructures: a Metadata Framework and Geo-portal Implementation.

2011-09-16T17:47:00.001-06:00

Long-term Preservation for Spatial Data Infrastructures: a Metadata Framework and Geo-portal Implementation. Arif Shaon, Andrew Woolf. D-Lib Magazine. September/October 2011.
Geospatial data is increasing, particularly with diverse environmental datasets. Long-term preservation of the data is not typically addressed, but it is very important for current and future use. Sustained access to environmental data is becoming more important and more difficult because it is increasing so dramatically.
Without effective long-term preservation, the data face the risk of becoming unusable over time. This article looks at the requirements, particularly metadata, for preserving this data. The authors have implemented a web-based portal prototype that demonstrates some functions of a preservation interface, such as data discovery using geospatial metadata, data downloading, metadata creation and validation. There is more to be done in this area.

Preserving Your Personal Digital Memories

2011-09-13T02:23:00.000-06:00

This is a free online course on preserving your personal digital materials: photos, documents, and other media. These are fragile and require special care to keep them useable. But preserving digital information is a new concept that most people have little experience with. As new technologies appear for creating and saving our personal digital information, older ones become obsolete, making it difficult to access older content. Learn about the problem with digital preservation of materials and hear about some simple, practical tips and tools to help you keep digital files safe.

Evaluating Open Source Digital Preservation Systems: A Case Study.

2011-09-13T02:10:00.014-06:00

Evaluating Open Source Digital Preservation Systems: A Case Study. Angela Jordan. Practical E-Records. August 18, 2011.
The University of Illinois Archives has implemented the “Practical E-Records Method,” a project that provides recommendations to help make digital curation and digital preservation systematic institutional functions. They tested Archivematica, which is essentially "an Ubuntu (Linux) distribution with extensions to support digital preservation actions using a web-based preservation dashboard." The test "started with elementary electronic records such as Microsoft Office documents and PDFs, then moved to complicated, larger file types, such as audio-visual objects." Some parts worked well, but there were a number of errors. "Given the immediate needs of the University Archives, the developing state of Archivematica, and other digital preservation development work taking place within the University Library, we chose not to incorporate the current version into our electronic records work flow."

There were three remaining concerns:

smaller institutions may lack the hardware or the technological capability to support the system
the installation process is not user friendly
the software is best run from a dedicated virtual server, to which many institutions may not have access. Running Archivematica on a dedicated virtual machine requires significant help from IT

The technological ability needed to successfully install and run this system is currently beyond the people who might benefit most. Once some of the issues are worked out in upcoming versions, Archivematica will be useful for smaller institutions that have less IT support than a large research library.