Moving Past ‘One Size Fits All’: Developing a Trajectory Deviance Index for Dynamic Measurement Modeling

Abstract

Dynamic Measurement Modeling (DMM) is a recently developed measurement framework for gauging developing constructs (e.g., learning capacity) that conventional single-timepoint tests cannot assess. Like most measurement models, overall model fit indices of DMM do not indicate the measurement appropriateness for each included student. For this reason, other measurement modeling paradigms (e.g., Item-Response Theory; IRT) utilize person-fit or model appropriateness statistics to indicate whether a measurement model appropriately describes the data from each individual student. However, within the extant DMM framework, no statistical index has yet been developed for this purpose. Thus, the current project advanced a person-specific DMM Trajectory Deviance Index (TDI) that captures the aberrance of an individual’s growth from the model-implied trajectory. Two simulation studies were conducted to examine and compare the distributional properties and effectiveness of four TDI candidates with different formulations. Consequently, the best functioning one was determined as the final formulation of the TDI. The data generation model was based on the parameter estimates from the Technology-enhanced, Research-based, Instruction, Assessment, and professional Development (TRIAD) cluster-randomized experiment data, which contains seven waves of mathematics test scores for students from pre-school through Grade 5. Besides the simulation work, an empirical study was also conducted to demonstrate the uses of the developed TDI within those real-world data. The results indicated that bilingual status was significantly related to the deviance of growth in early mathematics, whereas the other examined factors (i.e., intervention, age, gender, special education status, Socioeconomic status) were not. Incorporating TDI into DMM analysis strengthened the validity of score use and interpretation, and offered a quantitative means of determining which students in the dataset were not adequately served by the dynamic measurement model

    Similar works