Get the Most from Your Survey: An Application of Rasch Analysis for Education Leaders

High-quality measurement tools are critical to school improvement efforts. Education researchers frequently employ surveys in order to assess a host of variables associated with school improvement. This article asserts that Rasch modeling techniques enhance the quality of a measurement tool because they comprise elements of both qualitative and quantitative research approaches, and because Rasch modeling corrects the erroneous conclusions that result from the errors associated with ordinal response scale data. This article illustrates, with specific attention to the needs of education leaders and researchers, how the Rasch measurement model gauges the usefulness of survey instruments. This study illustrates the benefits of Rasch modeling using the scale that measures teacher external political efficacy (TEPE). Findings show that a set of four items captures this domain well

A Simulation Study of Missing Data with Multiple Missing X’s

When exploring missing data techniques in a realistic scenario, the current literature is limited: most studies only consider consequences with data missing on a single variable. This simulation study compares the relative bias of two commonly used missing data techniques when data are missing on more than one variable. Factors varied include type of missingness (MCAR, MAR), degree of missingness (10%, 25%, and 50%), and where missingness occurs (one predictor, two predictors, or two predictors with overlap). Using a real dataset, cells are systematically deleted to create various scenarios of missingness so that parameter estimates from listwise deletion and multiple imputation may be compared to the true estimates from the full dataset. Results suggest the multiple imputation works well, even when the imputation model itself is missing data. Accessed 7,222 times on https://pareonline.net from August 12, 2014 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right

Assessing essential unidimensionality of real data

The capability of DIMTEST in assessing essential unidimensionality of item responses to real tests was investigated. DIMTEST found that some test data fit an essentially unidimensional model and other data did not. Essentially unidimensional test data identified by DIMTEST then were combined to form two-dimensional test data. The power of Stout’s statistic T was examined for these two-dimensional data. DIMTEST results on real tests replicated findings from simulated tests-T discriminated well between essentially unidimensional and multidimensional tests. T was also highly sensitive to major traits and insensitive to relatively minor traits that influenced item responses. Index terms: DIMTEST, essential unidimensionality, essential independence, multidimensionality, unidimensionality

Refinement of Stout's Procedure for Assessing Latent Trait Unidimensionality

105 p.Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 1987.Currently, the most popular method of measuring an individual's ability is based on Item Response Theory (IRT). One of the most critical assumptions on which this IRT methodology is based is "unidimensionality". In general this assumption means that the items measure one and only one dimension or ability. In practice, however, this assumption cannot be strictly met because a multitude of factors influence test performance. What is actually required is that only one dominant trait influence test performance as a whole.Stout (1982) has developed a statistical significance test procedure for assessing the unidimensionality of a set of items consistent with the notion of counting the number of dominant traits being measured. The purpose of this thesis has been to complete a detailed investigation and modification of Stout's procedure based on theoretical and empirical reasoning.It was shown that unacceptably large bias occurs in the value of the statistic when most of the items in a test have high discrimination power. Three methods were proposed to correct for this bias, out of which one method, called Difficulty Check, worked. An algorithm was developed to determine the size of the subtests needed to compute Stout's statistic and a nonparametric index called AHAT has been developed as a crude estimate of item discrimination parameter and compared with other indices in the literature.Based on Monte Carlo simulations it was observed that the bias was completely eliminated. The procedure, in the case of unidimensionality, adheres to the desired level of significance, and the power of the statistical test, in the case of multidimensionality, is very good even when the correlation between the abilities is as high as 0.5.Based on this study, it is evident that Stout's procedure can be used for assessing unidimensionality of tests in many practical situations. Further research is recommended regarding the application of Stout's procedure for other purposes of educational measurement like detecting test bias and test equating.U of I OnlyRestricted to the U of I community idenfinitely during batch ingest of legacy ETD

Bayesian Prior Choice in IRT Estimation Using MCMC and Variational Bayes

This study investigates the impact of three prior distributions: matched, standard vague, and hierarchical in Bayesian estimation parameter recovery in two and one parameter models

Bayesian Prior Choice in IRT Estimation Using MCMC and Variational Bayes

Publisher's PDFThis study investigated the impact of three prior distributions: matched, standard vague, and hierarchical in Bayesian estimation parameter recovery in two and one parameter models. Two Bayesian estimation methods were utilized: Markov chain Monte Carlo (MCMC) and the relatively new, Variational Bayesian (VB). Conditional (CML) and Marginal Maximum Likelihood (MML) estimates were used as baseline methods for comparison. Vague priors produced large errors or convergence issues and are not recommended. For both MCMC and VB, the hierarchical and matched priors showed the lowest root mean squared errors (RMSEs) for ability estimates: RMSEs of difficulty estimates were similar across estimation methods. For the standard errors (SEs), MCMC-hierarchical displayed the largest values across most conditions. SEs from the VB estimation were among the lowest in all but one case. Overall, VB-hierarchical, VB-matched, and MCMC-matched performed best. VB with hierarchical priors are recommended in terms of their accuracy, and cost and (subsequently) time effectiveness.University of Delaware, School of Educatio