Scholarly Productivity and Impact: Developing a Quantifiable, Norm-based Benchmarking Methodology for Academic Emergency Medicine

Abstract

BACKGROUND: Quantifying and benchmarking scholarly productivity of emergency medicine faculty is challenging. While performance indicators including publication and citation counts are available, use of indicators to create normative references has lagged. The authors developed methodology to benchmark emergency medicine academician scholarly productivity (e.g., publications over time) and impact (e.g., citations per publication over time) against an appropriate reference group. METHODS: The methodology includes: 1) define time frame and scholarly metrics; 2) identify representative population; 3) reconcile alternative author names; 4) use analytic tool to identify scholarly output; 5) build database containing metrics; and 6) create benchmarking statistics, including subsamples. This study included emergency medicine faculty from 2011 to 2015, with total peer-reviewed publications and citations per publication as scholarly metrics. RESULTS: In the United States at the time of the search (2016) there were 200 academic emergency departments, 186 with public faculty listings, which yielded 6,727 academicians. For each academician, the authors calculated statistics about peer-reviewed publications and average citations per publication from 2011 to 2015 and created benchmarking rulers using percentile ranks. Productivity by year of graduation with terminal degree was compared within each subsample, finding that newly graduated faculty demonstrated higher productivity than their within-rank peers who graduated earlier. Finally, benchmarking tables were created that allow comparison of peer-reviewed publication counts and citations per publication for individual academicians against the norm. CONCLUSIONS: This benchmarking method can serve as a model for norm-based scaling of scholarly productivity for emergency medicine. This has important implications for performance review, promotion and hiring, and evaluating group productivity