Rhodes, Sarah

English

Georgetown Law Scholarly Commons

H3: Here’s to Your (Digital Archive’s) Good Health:Applying TRAC and Other Evaluation Parameters to a Local Digital Preservation Project102nd AALL Annual Meeting & ConferenceWashington, D.C.July 28, 2009 Speaker: Sarah RhodesAgenda• Introduction & Overview of The Chesapeake Project Legal Information Archive• Self-Assessment Evaluation Parameters & Findings• TRAC Assessment & Findings• Discussion of Costs & Lessons Learned About The Chesapeake Project• The Chesapeake Project is a shared Legal Information Archive– Two-year pilot (2007-2009) to investigate the feasibility of establishing a collaborative digital archive, shared by multipleinstitutions in the law library community, for the preservation of Web-published legal materials– Pilot Participants:• Georgetown Law Library• Maryland State Law Library• Virginia State Law Library– Affiliated with the Legal Information Preservation Alliance (LIPA)Digital Preservation System• Began harvesting/archiving Web content in early 2007 using OCLC Digital Archive • July 2008 = migration of archived files to a new two-tiered digital-preservation and access system– Access copy in CONTENTdm + archival masters in dark Digital Archive (similar to original OCLC Digital Archive)– Added point of access through CONTENTdm interface at www.legalinfoarchive.org, Web search engine discoveryAccess via local OPACAccess via WorldCat.orgAccess via subscription OCLC WorldCat databaseAccess via CONTENTdmSearch Engine DiscoveryProject Evaluation (Self-Assessment)• Evaluation to occur at one-year mark and two-year mark (end of pilot phase)• Quantitative/Objective Evaluation Parameters:– No. of items/titles archived during project’s first year – Analysis of archiving activity– Access statistics– ‘Link rot’ analysis, a count of archived items altered/removed from original locations on Web• Qualitative/Subjective Evaluation Parameters:– Staffing requirements– Time committed to project activities– Challenges & problems encountered – Progress toward the realization of the project’s mission, visionProject Evaluation (Self-Assessment)Preparation for Final Pilot Evaluation• First-Year Evaluation (self-assessment) conducted in March 2008• Follow-up 2009 Second-Year Evaluation based upon same quantitative/qualitative parameters• Also enlisted Center for Research Libraries (CRL) to conduct independent assessment based on criteria set forth in Trustworthy Repositories Audit & Certification (TRAC): Criteria and ChecklistFinal Pilot Project Evaluation, June 2009• Findings:– 4,306 digital items archived over two years– Usage spiked following migration to CONTENTdm/Digital Archive system:• March 2007-June 2008 = 6,612 instances of access • July 2008-February 2009 = 177,152 instances of access, 73,614 terms searchedFinal Pilot Project Evaluation, June 2009• Link Rot Analysis:– 2008 sample: • Link rot found in 8.3% of titles– Same sample, one year later: • Link rot found in 14.3% of titlesFinal Pilot Project Evaluation, June 2009• Qualitative analysis:– 2-25 hours devoted per week; task requiring most time = cataloging – Challenges: change of system, loss of our project’s visionary, Bob Oakley– Strong sense that mission accomplished throughout two-year pilot phase; vision is within reachTRAC Assessment Parameters• Three aspects of the project assessed, based on TRAC:– Organization (financial and operational framework and policies)– Preservation Strategy (processes and procedures governing management of archived digital objects)– Technology (assessment of OCLC’s system architecture, hardware, and software)• Areas of risk identified, recommendations providedTRAC Assessment Process• Comprehensive collection of project documentation provided to CRL Analyst• CRL Analyst site visit to Washington, D.C., and participant observation during February 2009 quarterly meeting• Two-day site visit to OCLC facilities in Dublin, Ohio, by CRL Analyst and Repository Architecture Technology AdvisorTRAC Assessment Findings• Project organization commended– “Overall, The Chesapeake Project provides good stewardship of the Web content it has identified and collected.”– Project addresses a real need– Project activities are “cost-effective and focused”– “Project decision- and policymaking apparatus is relatively lean and structured in a way that should ensure the archives’ responsiveness to the law library community.”TRAC Assessment Findings• Three areas of risk identified:– “Bit preservation” service may result in future difficulties associated with long-term preservation– Selection criteria and preservation strategies must evolve to adapt to dynamic “Web 2.0” as well as future Web-based technologies – To accommodate growth of project size and scope, base of support should be broadened/diversified, and commitments formalizedTRAC Assessment Findings• Recommendations fell within two general categories:– Relating to collaboration, e.g.:• Enlargement of participant population• Formalization of the partnership– Relating to “life cycle” management, e.g.:• Exploration of “current and future uses” of digital archive collections and “life-cycle” model of information preservationTRAC Assessment Costs• Scaled to accommodate smaller project, modest cost divided equally by three participating institutions• Did not include comprehensive technical audit of OCLC systems, but did include assessment of:– self-reported information from OCLC– third-party information about OCLC systems, and – an examination of a 10% random sample of preservation metadata records for archived digital objectsResponse to TRAC Assessment• Overall, very pleased, a worthwhile investment• Feel that some risks applied to the entire field of digital preservation and could be put into larger context• On-site OCLC visit, documentation review, and archive metadata test sample results affirm choice of OCLC for the project• Project expansion and diversification has been incorporated as a major goal in the post-pilot phaseLessons Learned• Define self-assessment parameters based on your project’s unique mission and goals; explore objective as well as subjective assessment measures• TRAC Criteria & Checklist provides significant and detailed guidance about best practices in digital preservation and should be consulted in self-auditing exercisesLessons Learned• Maintain thorough documentation relating to project policies, staffing, budgets, procedures and workflows, decision-making processes, and meetings• Maintain monthly reports of project activity and usage statistics • Keep and document data and samples used for evaluation purposes. These can be revisited in future assessments to measure change, progressLessons Learned• Be flexible in the reporting and presentation of your statistics. Technology is rapidly evolving, and inconsistencies in what you can and cannot measure are likely to occur• An independent, third-party assessment is a worthy investment, and can be especially helpful in identifying risks associated with offsite vendor systems. Negotiate scaled assessment costs to meet the needs of your project Lessons Learned• Understand and attempt to meet established standards and best practices  while also making independent decisions and adjustments appropriate to your project, preservation system, mission, priorities, and parent institutionMore Information• The Chesapeake Project Legal Information Archive: www.legalinfoarchive.org• Final Pilot Project Evaluation:www.legalinfoarchive.org/policies/legal_twoyearprojectevaluation_june2009.pdf• The Legal information Preservation Alliance (LIPA): www.aallnet.org/committee/lipa• TRAC Criteria & Checklist (from CRL): www.crl.edu/PDF/trac.pdfThank You!• Sarah Rhodes, Digital Collections LibrarianGeorgetown Law LibraryOffice: (202) 662-4065E-mail: sjr36@law.georgetown.edu

Here\u27s to Your (Digital Archive\u27s) Good Health: Applying TRAC and Other Evaluation Parameters to a Local Digital Preservation Project

NELLCO Consortium Legal Scholarship Repository

bepress Legal Repository

https://scholarship.law.georgetown.edu/cgi/viewcontent.cgi?article=1000&amp;context=digitalpreservation_presentations

Here\u27s to Your (Digital Archive\u27s) Good Health: Applying TRAC and Other Evaluation Parameters to a Local Digital Preservation Project

Abstract

Similar works

Full text

Available Versions

Georgetown Law Scholarly Commons

NELLCO Consortium Legal Scholarship Repository

bepress Legal Repository