Why the Major Field Test in Business Does Not Report Subscores: Reliability and Construct Validity Evidence Why the Major Field Test in Business Does Not Report Subscores: Reliability and Construct Validity Evidence

Abstract To assess the value of individual students' subscores on the Major Field Test in Business (MFT Business), I examined the test's internal structure with factor analysis and structural equation model methods, and analyzed the subscore reliabilities using the augmented scores method. Analyses of the internal structure suggested that the MFT Business measures a unidimensional construct, which does not support reporting individual students' subscores. Augmented score analyses revealed that the observed total score could approximate the true subscore more accurately than the observed subscores, which do not support reporting individuals' subscores either. The results from these two approaches provided consistent evidence in support of the current practice of not reporting individuals' subscores on the MFT Business. The relationship between the two approaches is discussed, followed by a recommendation of an alternate method for future research

A comparison of two procedures for constrained adaptive test construction

The relatively new shadow test approach (STA) to computerized adaptive testing (CAT) proposed by Wim van der Linden is a potentially attractive alternative to the weighted deviation algorithm (WDA) implemented at ETS. However, it has not been evaluated under testing conditions representative of current ETS testing programs. Of interest was whether STA would, under typical high-stakes on-demand testing situations, produce tests of comparable or better psychometric quality as those produced by the current weighted deviation algorithm. Based on simulated data, we found that the STA performed as well or slightly better than the WDA on two of the three commonly accepted testing objectives: measurement and content. The WDA appeared to perform slightly better than the STA when the issue is security or item exposure control. The paper provides a review of the rationale that led to the specific testing objectives employed, an outline of the test construction steps common to the two procedures investigated, and a description of the specific models and algorithms employed by both procedures. Detailed description of the simulation study conducted and the results obtained from both of the procedures for one of the pools are also provided. Results are summarized and further research needs in particular areas discussed