PDF of a powerpoint presentation from the 2014 Digital Preservation Meeting, Washington D. C., July 22-23, 2014.  Also available from Slideshare.https://digitalcommons.odu.edu/computerscience_presentations/1010/thumbnail.jp

Ainsworth, Scott G.

Brunelle, Justin F.

Kelly, Mat

Nelson, Michael L.

SalahEldeen, Hany

Weigle, Michele C.

English

Old Dominion University

Old Dominion UniversityODU Digital CommonsComputer Science Presentations Computer Science7-23-2014Assessing the Quality of Web ArchivesMichael L. NelsonOld Dominion University, mnelson@odu.eduScott G. AinsworthOld Dominion UniversityJustin F. BrunelleOld Dominion UniversityMat KellyOld Dominion UniversityHany SalahEldeenOld Dominion UniversitySee next page for additional authorsFollow this and additional works at: https://digitalcommons.odu.edu/computerscience_presentationsPart of the Archival Science CommonsThis Book is brought to you for free and open access by the Computer Science at ODU Digital Commons. It has been accepted for inclusion inComputer Science Presentations by an authorized administrator of ODU Digital Commons. For more information, please contactdigitalcommons@odu.edu.Recommended CitationNelson, Michael L.; Ainsworth, Scott G.; Brunelle, Justin F.; Kelly, Mat; SalahEldeen, Hany; and Weigle, Michele C., "Assessing theQuality of Web Archives" (2014). Computer Science Presentations. 8.https://digitalcommons.odu.edu/computerscience_presentations/8AuthorsMichael L. Nelson, Scott G. Ainsworth, Justin F. Brunelle, Mat Kelly, Hany SalahEldeen, and Michele C.WeigleThis book is available at ODU Digital Commons: https://digitalcommons.odu.edu/computerscience_presentations/8Digital Preservation, July 22-23, 2014, Washington DCAssessing the Quality of Web ArchivesMichael L. NelsonScott G. Ainsworth, Justin F. Brunelle, Mat Kelly, Hany SalahEldeen, Michele C. WeigleOld Dominion UniversityWeb Science & Digital Libraries Research Groupws-dl.cs.odu.edu@WebSciDLDigital Preservation, July 22-23, 2014, Washington DCThe State of Web Archiving"Hooray!  It's in the archive!"vs. "How well was it archived?"current:future:Digital Preservation, July 22-23, 2014, Washington DChttp://web.archive.org/web/20140717152222/http://vk.com/strelkov_infohttp://www.csmonitor.com/World/Europe/2014/0717/Web-evidence-points-to-pro-Russia-rebels-in-downing-of-MH17-videoDigital Preservation, July 22-23, 2014, Washington DChttp://web.archive.org/web/20140717152222/http://vk.com/strelkov_infohttp://www.csmonitor.com/World/Europe/2014/0717/Web-evidence-points-to-pro-Russia-rebels-in-downing-of-MH17-videoDigital Preservation, July 22-23, 2014, Washington DCThree Ways We're  Assessing Quality• Weighting the "importance" of missing embedded resources – "damage" measure for comparing archived pages• Detecting "temporal violations"– some rendered pages never existed• Defining an archival tool benchmark– "Archive Acid Test"Digital Preservation, July 22-23, 2014, Washington DCNot All Mementos Are Created Equal: Measuring The Impact Of Missing ResourcesJCDL 2014http://www.cs.odu.edu/~mln/pubs/jcdl-2014/jcdl-2014-brunelle-damage.pdfDigital Preservation, July 22-23, 2014, Washington DCM = 0.17D = 0.09(live web)M = 0.24D = 0.41(missing main)M = 0.29D = 0.36(missing logo + navigation)Synthetic Damage:Removing Images From xkcd.comdamage (D) differs from % missing (M)!Digital Preservation, July 22-23, 2014, Washington DCWas missing resourceimportant?  <img>and <embed>can leave hints about size and centrality.For CSS, we look at the distribution of background color in page divided intovertical thirds.Digital Preservation, July 22-23, 2014, Washington DCWeights from Turker Assessment of Damage first: establish that Turkers can determine damaged vs. undamaged pages (81% of the time)second: find weights that match Turker's rankings of (real) differently damaged versions of the same pageDigital Preservation, July 22-23, 2014, Washington DCGood News: Although M is steady/increasing, D is decreasingDigital Preservation, July 22-23, 2014, Washington DCA Framework for Evaluation of Composite Memento Temporal Coherence(in preparation)http://arxiv.org/abs/1402.0928Digital Preservation, July 22-23, 2014, Washington DCAs Presented by IAhttp://web.archive.org/web/20041209190926/http://www.wunderground.org/cgi-bin/findWeather/getForecast?query=50593 (now 404, but that's a different story…) Digital Preservation, July 22-23, 2014, Washington DCNot Everything Is 200412091900926+ 9 monthshttp://web.archive.org/web/20041209190926/http://www.wunderground.org/cgi-bin/findWeather/getForecast?query=50593 (now 404, but that's a different story…) Digital Preservation, July 22-23, 2014, Washington DChtml jpegjpegjpegConsider:<html><img src="foo.jpeg"></html>change change change changeDigital Preservation, July 22-23, 2014, Washington DChtml jpegjpegjpegCorrect Archival Renderingchange change change changeDigital Preservation, July 22-23, 2014, Washington DChtml jpegjpegjpegBut Archives Miss Updates…change missedchangechange changeDigital Preservation, July 22-23, 2014, Washington DChtml jpegjpegjpegYou Can Choose the Closestchange missedchangechange change(closest is the current policy of most archives)Digital Preservation, July 22-23, 2014, Washington DChtml jpegjpegjpegYou Can Choose the Pastchange missedchangechange changeDigital Preservation, July 22-23, 2014, Washington DChtml jpegjpegjpegOr You Can "Bracket" the HTML change missedchangechange change(when possible, brackets can be made via HTTP metadata or content comparison)?In this case, there is no right answer.Either choice will result in a temporal violation.Digital Preservation, July 22-23, 2014, Washington DCCompleteness vs. CoherenceDescriptionClosest Single ArchiveClosestMulti-ArchiveBracketSingle ArchiveBracketMulti-ArchiveCompletenessMean complete 76.1% 80.2% 76.2% 80.3%Mean missing 23.9% 19.8% 23.8% 19.7%Temporal CoherenceMean prima facie coherent 41.0% 40.9% 54.7% 54.6%Mean possibly coherent 27.3% 27.3% 12.8% 14.2%Mean probably violative 2.5% 5.3% 2.5% 5.3%Mean prima facie violative 5.3% 5.3% 6.2% 6.2%At least 5% of pages can be shown to be temporal violations Digital Preservation, July 22-23, 2014, Washington DCThe Archival Acid Test: Evaluating Archive Performance on Advanced HTML and JavaScriptJCDL 2014http://ws-dl.blogspot.com/2014/07/2014-07-14-archival-acid-test.htmlhttp://acid.matkelly.com/Digital Preservation, July 22-23, 2014, Washington DCInspired by the Acid3 Test for Browsershttp://acid3.acidtests.org/http://en.wikipedia.org/wiki/Acid3Digital Preservation, July 22-23, 2014, Washington DCThe Archival Acid TestHeritrix WARCreateGNU WgetArchiving ToolsArchivesDigital Preservation, July 22-23, 2014, Washington DCArchival Tools & Sites on Acid3Digital Preservation, July 22-23, 2014, Washington DCArchival Acid Tests Digital Preservation, July 22-23, 2014, Washington DCArchival Tools & Sites on AAT(mummify.it died in early 2014)Digital Preservation, July 22-23, 2014, Washington DCFuture of Web Archiving:Increasing Quantitative Analysis• Measure "damage" instead of completeness of archived pages– enables large-scale comparison of archives• Even if an embedded resource is present, it doesn't mean it's right– ~5% of archived pages have temporal violations• To improve the quality of the archives, we need to be able to benchmark archival tools– Archival Acid Test is an easy to use benchmark 

Assessing the Quality of Web Archives

https://digitalcommons.odu.edu/cgi/viewcontent.cgi?article=1010&amp;context=computerscience_presentations

Assessing the Quality of Web Archives

Abstract

Similar works

Full text

Available Versions

Old Dominion University