Holmes, Kristi L

Sarli, Cathy C

Digital Commons@Becker

Washington University School of MedicineDigital Commons@BeckerBecker Library Publications and Presentations Becker Medical Library2011Identifying and assessing research impactCathy C. SarliKristi L. HolmesFollow this and additional works at: http://digitalcommons.wustl.edu/becker_pubsPart of the Medicine and Health Sciences CommonsThis Presentation Poster is brought to you for free and open access by the Becker Medical Library at Digital Commons@Becker. It has been acceptedfor inclusion in Becker Library Publications and Presentations by an authorized administrator of Digital Commons@Becker. For more information,please contact engeszer@wustl.edu.Recommended CitationSarli, Cathy C. and Holmes, Kristi L., "Identifying and assessing research impact." 2011 Annual International Science of Team Science(SciTS) Conference, Chicago, IL. 2011. Paper 18.http://digitalcommons.wustl.edu/becker_pubs/18RESEARCH PROCESS & OUTPUTS New basic research studiesNew clinical studies Intellectual property commercializationInterdisciplinary collaboration with other institutionsIndustry partnershipsCost-effective health interventionImproved standard of carePolicy or legislationMultidisciplinary collaboration within an institutionClinical guidelines Clinically effective approach to management of a diseaseVaccinesQuality Measure GuidelineNew field of researchCPT CodesImproved quality of lifeEnhancement of existing resources and expertiseContinuing EducationCurriculum materialsStandardsMeasurement devicesImproved pharmaceuticalsImproved medical devicesIdentifying and Assessing Research ImpactCommunity partnershipsSoftwareNew experimental methodologiesAgency partnershipsBiologicsDatabasesDiusion represents knowledge transfer of research outputs into meaningful outcomes. Diusion is manifested in various pathways.In order to see the all the leaves on the tree, i.e., research impact, one must go beyond traditional citation analysis and metrics. DIFFUSION1 - Sarli CC, Dubinsky EK, and Holmes KL. 2010. Beyond citation analysis: A model for assessment of research impact. J Med Libr Assoc. January; 98(1): 17-23.       http://www.pubmedcentral.nih.gov/2801963 2 - Becker Medical Library Model for Assessment of Research Impact http://becker.wustl.edu/impact/assessment/index.html 3 - Wells R, Whitworth A. 2007. Assessing outcomes of health and medical  research: do we measure what counts or count what we can measure? Australia and  New        Zealand Health Policy, 4:14. http://www.anzhealthpolicy.com/content/4/1/14Advancement of Knowledge Economic Benet Community BenetClinical ImplementationLegislation and Policy  To understand the true impact of research, it is necessary to analyze the research study process in the clinical and biomedical research environment to identify tangible indicators of research impact that are not readily discernible using citation analysis.PATHWAYS TO RESEARCH IMPACTBiological materials BooksPharmaceutical PreparationsPatentsDatabasesMeasurement InstrumentsGray LiteratureOutreach EortsBiological MaterialsCollaborationsResearch MethodologiesBook ChaptersAdvancement of Knowledge represents research outputs and/or activities that contribute to the scholarly record.How was the research output used? How was awareness of research output demonstrated? • License agreement granted for use of intellectual property generated by the research study• Clinical data generated in support of marketing a medical device• Consensus Development Conference• Curriculum guidelines• Cited referencesClinical Implementation is the adoption of research outputs and/or activities in clinical applications.How was translation of research outputs into clinical applications demonstrated? • Creation of Current Procedural Terminology (CPT) codes • Medical device is registered with FDA and used in delivery of healthcare• Research study cited in private insurance benefit plan in support of coverage• Healthcare providers report that research outputs from a research study has aﬀected the way they manage       patientsCommunity Benet is the enhancement of community health outcomes as a result of research outputs.How was community health enhanced as a result of research output? • A blood test for detection of damage to the heart muscle leads to early treatment for myocardial infarctions, a       leading cause of death worldwide.• A critically appraised economic evaluation reports that research outputs from a research study demonstrates an       eective injury intervention strategy in reducing head injuries: “wearing a bicycle helmet reduced the risk of       incurring a head injury following an accident by 70%.”• The Ocular Hypertension Treatment Study (OHTS) demonstrated that medical treatment of people with       intraocular pressure (IOP) of > or =24 mm Hg reduces the risk of the development of primary open-angle       glaucoma (POAG) by 60%.Legislation and Policy Enactment represents codication of research outputs into public law, guidelines, standards or policy.How did research outputs inuence or result in enactment of public law, guidelines, standards or policy? • Smoke-free Air Acts • Automated External Defibrillators (AED) Acts• ANSI Standards• AIA Guidelines for Design and Construction of Health Care Facilities• Testimony given by a research investigator before a governing body supports creation of a bill and/or regulationEconomic Benet represents economic outcomes as a result of research outputs and/or activities.How did research outputs result in economic outcomes?• Discovery of lithium treatment for manic-depressive disorders saved the US over $145 billion in hospital costs. • Cochlear implants in profoundly or completely deaf children not only improve their quality of life but are also       expected to save society about $53,000 over the individual child's lifetime, including both direct and indirect       costs.General Examples of Research ImpactNew Standard of CareHutcheson, Kelly A.  Application of new ophthalmic technology in the pediatric patient: Pediatrics and strabismus. Current Opinion in Ophthalmology, September 2007;18(5):384–391. “In adult patients, measurement of the central corneal thickness (CCT) has become standard of care for patients with known or suspected glaucoma. In the Ocular Hypertension Treatment Study, the CCT was found to be a strong predictor for the development of glaucoma, with patients having corneas thinner than 555 µm having a three times greater risk than those with thicker corneas...it has become the standard of care to perform pachymetry measurements on all children with or suspected of having glaucoma.”New Diagnostic CriteriaThimons, J. James. Pachymetry: The New Standard of Care In Glaucoma. Optometric Management. May 2006.“Since the publication of the Ocular Hypertension Treatment Study (OHTS), central corneal thickness as a significant risk factor for glaucoma has been central to the discussion of its management. Pachymetry is a remarkable tool and has rapidly become a mainstay of clinical practice in the area of glaucoma. The question of its status as a standard-of care has, to a large degree, been answered by the clinical community. The American Academy of Ophthalmology deemed CCT an integral part of the glaucoma evaluation. The American Optometric Association incorporated CCT into its recommendations for Preferred Practice Patterns. Medicare issued a unique reimbursement code more rapidly than for any other technology in the history of ophthalmic science.” Quality Measure GuidelinesThe National Committee for Quality Assurance (NCQA), Health Plan Employer Data and Information Set (HEDIS)     HEDIS Glaucoma Screening in Older Adults (GSO) Measure (first year optional Senior measure), 2005.         HEDIS Glaucoma Screening in Older Adults (GSO) Measure (required Senior measure), 2006-2009. U.S. Preventive Services Task Force Evidence Synthesis Primary Care Screening for Ocular Hypertension and Primary Open-Angle Glaucoma: Evidence Synthesis, 2005. Based on USPSTF rating criteria, the quality of the OHTS study was rated “good.” The OHTS study provides. . . “good evidence that treatment for ocular hypertension and early POAG is beneficial.” Reviews Ohba N, Nakao K, Isashiki Y, Ohba A. The 100 most frequently cited articles in ophthalmology journals. Arch Ophthalmol. 2007 Jul;125(7):952-60.Included as one of the top 100 most frequently cited articles is: Kass MA, Heuer DK, Higginbotham EJ; et al. The Ocular Hypertension Treatment Study: a randomized trial determines that topical ocular hypotensive medication delays or prevents the onset of primary open angle glaucoma. Curriculum GuidelinesInternational Council on Ophthalmology     Principles and Guidelines for the Curriculum for Education of the Ophthalmic Specialist     Guidelines and Standards for Education of an Ophthalmologist: A Curricular Outline     Appendix: Glaucoma: Ocular Hypertension Treatment StudyRoyal College of Ophthalmologists     Guidelines for the Management of Open Angle Glaucoma and Ocular HypertensionNew Research Studies Corneal Endothelial Cell DensityR-01 Short Wavelength Automated Perimetry  R-01 Confocal Scanning Laser Ophthalmoscopy Pachymetry Ancillary StudyR-21 EGPS/OHTS Confocal Scanning Laser Ophthalmoscopy CollaborationOcular Hypertension Treatment Study (OHTS): Ancillary Genetic TestingR-21 European Glaucoma Prevention Study (EGPS)/OHTS Collaborative AnalysisClinical/Practice Guidelines American Academy of Ophthalmology      Primary Open-Angle Glaucoma Suspect Preferred Practice Pattern™ Guideline, 2005. American Optometric Association    Care of the Patient with Open Angle Glaucoma: Reference Guide for Clinicians, 2002. International Council of Ophthalmology/International Fed. of Ophthalmological Societies    Primary Open-Angle Glaucoma (Initial Evaluation), 2007.    Primary Open-Angle Glaucoma (Follow-up Evaluation), 2007.    Primary Open-Angle Glaucoma Suspect (Initial and Follow-up Evaluation), 2007.Royal College of Ophthalmologists    Guidelines for the Management of Open Angle Glaucoma and Ocular Hypertension, 2004.Measurement Instruments Devers Risk Calculator Glaucoma Risk Calculator—The Scoring Tool for Assessing Risk Pfizer Hand-Held Glaucoma Five Year Estimator Risk Calculator Glaucoma Five Year Estimator Risk CalculatorAmerican Medical Association Current Procedural Terminology (CPT) Codes CPT Code Category III for pachymetry established in 2002. CPT Code Category I for pachymetry established in 2004. “Corneal pachymetry (the measurement of corneal thickness) is an essential tool for glaucoma diagnosis and management because measurement of IOP is aﬀected by the thickness of the cornea. . .The determination of an accurate IOP improves the diagnosis, screening and management of patients with glaucoma and OHT.”   CPT Assistant: Authoritative Coding Information. The American Medical Association, July 2004. Change in Delivery of Healthcare Services Murphy, John. National Panel: OHTS aﬀects when optometrists treat glaucoma. Review of Optometry, July 15, 2003;140(7). “Three out of four panelists say that the Ocular Hypertension Treatment Study (OHTS) has aﬀected the way they manage glaucoma patients.”“We now routinely perform ultrasound pachymetry on all glaucoma suspects and ocular hypertensives.”“Three in 10 optometrists now perform baseline pachymetry on new glaucoma patients. Many say they have since ordered (or are about to order) a pachymeter.” “I treat patients with a family history and thinner corneas earlier than I would have before.” “Five years ago, only about one-third of optometrists typically treated glaucoma patients themselves. Now, two-thirds of optometrists say they treat glaucoma patients largely on their own.”  Cost-Eective Intervention Management of ocular hypertension: a cost-eﬀectiveness approach from the Ocular Hypertension Treatment Study. Kymes S, et al. Am J Ophthal 2006 Jun;141(6):997-1008.“The Ocular Hypertension Treatment Study (OHTS) demonstrated that medical treatment of people with intraocular pressure (IOP) of > or =24 mm Hg reduces the risk of the development of primary open-angle glaucoma (POAG) by 60%. Although the treatment of individual patients is largely dependent on their attitude toward the risk of disease progression and blindness, the treatment of those patients with IOP of > or =24 mm Hg and a > or =2% annual risk of the development of glaucoma is likely to be cost-eﬀective.”Private and Public Healthcare Benet Plans Many private and public insurance benefit plans allow for pachymetry for measurement of corneal thickness, with variations in coverage.Example: Cigna: “The OHTS established corneal thickness as a risk factor for glaucoma.”Consensus Development Conferences Canadian Perspectives in Glaucoma Management: Setting Target Intraocular Pressure Range Can J Ophthalmol. 2003 Apr;38(3):189-97. “Recent evidence supports the concept of aggressively lowering IOP to help prevent glaucoma related blindness.” OHTS is referred  to as one of the “gold standard” prospective randomized clinical trials that support this concept.”Continuing Education Materials10th Annual Glaucoma Report: High Tech Glaucoma Devices Hasten Diagnosis, 2004.“Perhaps the most important point: The Ocular Hypertension Treatment Study (OHTS) identified corneal thickness not just as a variable that aﬀects Goldmann IOP readings but as an independent risk factor for developing glaucoma. Specifically, OHTS uncovered a linear relationship between IOP and corneal thickness in patients with "thicker" corneas.” 13th Annual Glaucoma Report: Calculate the Risk Factors For Glaucoma, 2007.“. . . the Ocular Hypertension Treatment Study (OHTS) and other studies published during the last decade have fundamentally changed how we diagnose and manage glaucoma, and have helped us to identify such risks. In past years, we diagnosed open-angle glaucoma based on elevated intraocular pressure (IOP), visible optic nerve damage and correlated visual field loss. ”Research Impact: The Ocular Hypertension Treatment Studyhttp://ohts.wustl.edu/“It is no longer enough to measure what we can – we need to measure what matters.”-- Wells & Whitworth Cathy C. Sarli, MLIS, AHIP  and Kristi L. Holmes, PhD     Translational Research Support Division, Bernard Becker Medical Library, Washington University in St. Louis School of Medicine, St. Louis, MO; Washington University Institute of Clinical and Translational SciencesThe Becker Medical Library Model for Assessment of Research Impact       provides indicators of evidence of impact based on resulting diusion of research outputs and activities, resources for locating evidence of impact, and also includes strategies that can be utilized by biomedical scientists to enhance their research impact.The recent emphasis in demonstrating translational outcomes of clinical and biomedical research ndings into clinical practice and community benet has spurred a need for new methods beyond traditional citation metrics to document the impact of research. The Becker Medical Library Model for Assessment of Research Impact can assist with the discovery process by providing a framework to document research outputs and activities, identify collaborative eorts, and assess the diusion of knowledge and resulting research impact for an individual author or research group.1,2SoftwareWebsitesWikisInvited  PresentationsConference AbstractsJournal ArticlesContributed  Presentations

Identifying and assessing research impact

http://digitalcommons.wustl.edu/cgi/viewcontent.cgi?article=1017&context=becker_pubs

Identifying and assessing research impact

Abstract

Similar works

Full text

Available Versions

Digital Commons@Becker