Multilevel IRT Modeling in Practice with the Package mlirt

Variance component models are generally accepted for the analysis of hierarchical structured data. A shortcoming is that outcome variables are still treated as measured without an error. Unreliable variables produce biases in the estimates of the other model parameters. The variability of the relationships across groups and the group-effects on individuals' outcomes differ substantially when taking the measurement error in the dependent variable of the model into account. The multilevel model can be extended to handle measurement error using an item response theory (IRT) model, leading to a multilevel IRT model. This extended multilevel model is in particular suitable for the analysis of educational response data where students are nested in schools and schools are nested within cities/countries.\u

Multilevel IRT Modeling in Practice with the Package mlirt

Variance component models are generally accepted for the analysis of hierarchical structured data. A shortcoming is that outcome variables are still treated as measured without an error. Unreliable variables produce biases in the estimates of the other model parameters. The variability of the relationships across groups and the group-effects on individuals' outcomes differ substantially when taking the measurement error in the dependent variable of the model into account. The multilevel model can be extended to handle measurement error using an item response theory (IRT) model, leading to a multilevel IRT model. This extended multilevel model is in particular suitable for the analysis of educational response data where students are nested in schools and schools are nested within cities/countries.

Item Response Theory for Peer Assessment

As an assessment method based on a constructivist approach, peer assessment has become popular in recent years. However, in peer assessment, a problem remains that reliability depends on the rater characteristics. For this reason, some item response models that incorporate rater parameters have been proposed. Those models are expected to improve the reliability if the model parameters can be estimated accurately. However, when applying them to actual peer assessment, the parameter estimation accuracy would be reduced for the following reasons. 1) The number of rater parameters increases with two or more times the number of raters because the models include higher-dimensional rater parameters. 2) The accuracy of parameter estimation from sparse peer assessment data depends strongly on hand-tuning parameters, called hyperparameters. To solve these problems, this article presents a proposal of a new item response model for peer assessment that incorporates rater parameters to maintain as few rater parameters as possible. Furthermore, this article presents a proposal of a parameter estimation method using a hierarchical Bayes model for the proposed model that can learn the hyperparameters from data. Finally, this article describes the effectiveness of the proposed method using results obtained from a simulation and actual data experiments

India shining and Bharat drowning: comparing two Indian states to the worldwide distribution in mathematics achievement

This paper uses student answers to publicly released questions from an international testing agency together with statistical methods from Item Response Theory to place secondary students from two Indian states -Orissa and Rajasthan -on a worldwide distribution of mathematics achievement. These two states fall below 43 of the 51 countries for which data exist. The bottom 5 percent of children rank higher than the bottom 5 percent in only three countries-South Africa, Ghana and Saudi Arabia. But not all students test poorly. Inequality in the test-score distribution for both states is next only to South Africa in the worldwide ranking exercise. Consequently, and to the extent that these two states can represent India, the two statements"for every ten top performers in the United States there are four in India"and"for every ten low performers in the United States there are two hundred in India"are both consistent with the data. The combination of India's size and large variance in achievement give both the perceptions that India is shining even as Bharat, the vernacular for India, is drowning. Comparable estimates of inequalities in learning are the building blocks for substantive research on the correlates of earnings inequality in India and other low-income countries; the methods proposed here allow for independent testing exercises to build up such data by linking scores to internationally comparable tests.Secondary Education,Educational Sciences,Teaching and Learning,Primary Education,Tertiary Education