Does DIF signal a lack of essential unidimensionality?

Abstract

The majority of applied differential item functioning (DIF) studies test hypotheses regarding probability differences. The presence of DIF then is often attributed to the assumption of multidimensionality, without any empirical evidence of a demonstrable additional dimension when all that has been found is that the grouping variable influenced measurement, which any number of rival hypotheses might explain. Therefore, it is of interest to know whether DIF can manifest itself as multidimensionality through a dimensionality assessment procedure. Specifically, the current study investigated under what conditions a DIF-contaminated test could display a lack of essential unidimensionality via DIMTEST. DIF data were modeled using two simulation methods. One used Raju’s (1988) formula (item characteristic curve, ICC area difference) and the other used Shealy and Stout’s (1993a, 1993b) multidimensional model for DIF (MMD). A variety of factors were manipulated in the study (e.g., DIF magnitude, percentage of DIF items). Additionally, SIBTEST (Simultaneous Item Bias Test) was used to examine DIF conditions (modeled by both methods) with similar DIMTEST rejection rates to determine whether a MMD based DIF detection method can correctly identify known DIF items modeled by both the area difference and MMD methods. The study results suggest that a test can be contaminated with detectable DIF items but still hold essential unidimensionality. It is hoped that the study results can motivate psychometricians to revisit theoretically/methodologically/analytically/scientifically the appealing conceptualization of DIF as signaling multidimensionality and ponder important questions such as whether the influence of an external variable -group membership- equals the presence of multidimensionality, or whether rival hypotheses other than multidimensionality might explain DIF, thus affecting the very way in which DIF is conceptualized and interpreted in both applied and methodological studies, potentially having an impact on millions of test takers