Challenges of Long-Term Digital Archiving: A Survey

Abstract

With an ever-increasing volume of digital records and compliance requirements mandated by regulations, electronic record archiving grows to be more and more important in the digital era. The fundamental functionality of digital archiving includes keeping data content intact and providing provable evidence of events ever happened to the data. The main challenges of long-term digital archiving include: 1)authenticity and integrity of data content; 2)viability of information due to technology obsolescence; 3)reliable, affordable, sustainable and efficient archival media. All modifications to a digital archiving system should be authenticated properly. Authenticity is not enough to protect archived data from human errors or malicious attacks, various redundancy techniques are used to protect data integrity. Furthermore, it is difficult to correctly interpret data created by legacy hardware/software infrastructure on current computing platform as people and organizations are using increasingly complex software tools, data models and semantics, where related formats, standard and semantics are evolving quickly. Standard models and formats are proposed to mitigate the obsolesce problem. For long-term preservation purpose, it is also desirable that the archival media is reliable, affordable, sustainable and efficient. As the size of a single magnetic disk keeps growing to be Tera-scale or even Peta-scale with plumping per-byte cost, magnetic devices become a promising candidate for long-term digital preservation. However, the uncorrectable corruption rates(UER) of 1 bit corruption in 1 Terabyte 1 to 1 bit corruption in 100 Terabyte pose a challenge to the archiving system as the bit corruption may stay unnoticed for months. We propose several strategies to address this problem: checksumming, replication and efficient au