European Educational Research Quality Indicators (EERQI): A first prototype framework of intrinsic and extrinsic indicators

Abstract

Mooij, T. (2011, 15-16 March). European Educational Research Quality Indicators (EERQI): A first prototype framework of intrinsic and extrinsic indicators. Paper presented at the final EERQI conference, Brussels, University Foundation.The goal of the project ‘European Educational Research Quality Indicators’ (EERQI) is to improve upon citation-only based assessments of the quality or impact of educational and other research. In this conference paper two research questions will be answered: 1) What are the results of applying the ‘intrinsic criteria’ rigour, originality, significance, integrity, and style, in a peer review evaluation of educational research articles? 2) What are the results of analysing the relationships between both intrinsic and extrinsic indicators, to construct an example of a prototype EERQI framework? The first question is answered by a pilot in which 78 peer reviewers assess 117 research documents according to rigour, originality, significance, integrity, style, and miscellaneous. In a second pilot, peer reviewers evaluate 20 intrinsic indicators of 177 research documents written by 268 authors. Extrinsic indicators of the same documents contain various types of web-based information. Structural model analysis using Mplus results in a measurement model with three intrinsic and two extrinsic latent factors. The outcomes support the hypothesis that the more the reviewed document is related to the reviewer‘s own area of research, the higher the document is evaluated with respect to 1) significance, originality and consistency and 2) methodological adequacy. No relationships exist between the reviewer‘s own area of research and the two extrinsic factors. These effects may reflect some subjective evaluation bias that seems to take place in peer reviewing. The outcomes support the validity of the EERQI conceptual framework and the related research.This European project was funded by the EC Seventh Framework Programme, under the Socio-Economic Sciences and Humanities Theme (SSH