Measurement Error in Research on Human Resource Decisions and Firm Performance: How Much Error is There and How Does its Influence Effect Size Estimates?
Recent empirical research finds that the relationship between human resource (HR) decisions and firm performance is significant in both statistical and practical terms. However, the typical research design in this area relies upon on a single respondent to validly assess firmwide HR practices. To date, no study has adequately addressed the reliability of such measures, a basic requirement of construct validity. Previous efforts have either defined reliability so narrowly as to miss a major source of measurement error (raters) or have estimated the unreliability due to raters using incorrect methods. In both cases, the result is upwardly biased estimates of reliability. We estimate reliabilities using intraclass correlation and generalizability coefficients. Our reliability estimates suggest substantial measurement error in the types of HR effectiveness and HR practice measures typically used to predict firm performance. We discuss how this degree of measurement influences research and policy implications
