This report discusses the Comparative Assessment of Peer Review (CAPR) project outcomes. The authors investigate the review process for agencies that fund scientific research

Burggren, Warren W.

Frodeman, Robert

Holbrook, J. Britt

Mitcham, Carl

Moen, William E.

UNT Digital Library

Comparative Assessment of Peer Review: Project Outcomes Report  NSF Award #0830387 Principal Investigator: Robert Frodeman Co-PIs: James Britt Holbrook, William Moen, Warren Burggren, Carl Mitcham Public funding agencies are required to demonstrate accountability to their government funders (e.g., Congress) as well as to the public.  Some agencies – including the US National Science Foundation (NSF) – have used broader societal impacts criteria as part of the review process of grant proposals in order to connect scientific research to societal needs. But these agencies have often encountered questions from scientists and engineers for how to integrate such demands for broader societal impacts into their research proposals. In an effort to help clarify the idea of broader impacts, in 2010 NSF and Congress proposed a list of national needs that NSF-funded research would be required to meet. But was this the best solution to the problem? Our research asked several questions: Why do scientists express confusion at – and sometimes resist – the requirement to address the societal impacts of their research? Was it because the broader impacts criterion was unclear? Was it because they felt like they lacked the expertise to judge anything beyond the science for its own sake? Or was it because they felt that such requirements interfered with scientific autonomy? Our research suggested that it was less that scientists and engineers felt unqualified to propose or judge broader impacts and more that they simply did not want to address the broader impacts of their research. This suggested that a focus on clarifying broader impacts activities may be misdirected. Researchers may complain that broader impacts criteria are unclear not because the criteria are in fact unclear, but rather because they do not want to address the criteria in the first place.  Dividing questions of ‘broader impacts’ off from questions of ‘intellectual merit’ also creates the impression that such impacts on society are secondary to impacts on science. But broader impacts are an essential facet of research. Benefits to society, in fact, are what drive science funding. Instead of providing a checklist – national needs might change rapidly and unpredictably, after all – we ought to require scientists and engineers to think about how their research responds to pressing societal needs. We need, in other words, to balance the value of scientific autonomy with the value of societal accountability. Our findings were incorporated into a recent review and revision of NSF's Merit Review Criteria, which identified the need to: • Retain freedom for researchers to posit their own broader impacts rather than relying on a predetermined list of national needs • Treat broader impacts more like intellectual merit, and  • Develop better ways to measure the broader impacts of research The CAPR website, which includes a digital repository containing over 850 documents related to peer review is accessible here: http://csid-capr.unt.edu/.  

Comparative Assessment of Peer Review: Project Outcomes Report

https://digital.library.unt.edu/ark:/67531/metadc139466/m2/1/high_res_d/CAPR_Project_Outcomes_Report.pdf

Comparative Assessment of Peer Review: Project Outcomes Report

Abstract

Similar works

Full text

Available Versions

UNT Digital Library