Not All Mementos Are Created Equal: Measuring The Impact Of Missing Resources

Abstract

Web archives do not capture every resource on every page that they attempt to archive. This results in archived pages missing a portion of their embedded resources. These em-bedded resources have varying historic, utility, and impor-tance values. The proportion of missing embedded resources does not provide an accurate measure of their impact on the Web page; some embedded resources are more important to the utility of a page than others. We propose a method to measure the relative value of embedded resources and as-sign a damage rating to archived pages as a way to evaluate archival success. In this paper, we show that Web users’ perceptions of damage are not accurately estimated by the proportion of missing embedded resources. The proportion of missing embedded resources is a less accurate estimate of resource damage than a random selection. We propose a damage rating algorithm that provides closer alignment to Web user perception, providing an overall improved agree-ment with users on memento damage by 17 % and an im-provement by 51 % if the mementos are not similarly dam-aged. We use our algorithm to measure damage in the Inter-net Archive, showing that it is getting better at mitigating damage over time (going from 0.16 in 1998 to 0.13 in 2013). However, we show that a greater number of important em-bedded resources (2.05 per memento on average) are missing over time