A maximin model for test design with practical constraints

A "maximin" model for item response theory based test design is proposed. In this model only the relative shape of the target test information function is specified. It serves as a constraint subject to which a linear programming algorithm maximizes the information in the test. In the practice of test construction there may be several demands with respect to the properties of the test. The way in which these can be formulated as linear constraints in the model is demonstrated. The constraints discussed include: (1) test composition; (2) administration time; (3) selection of item features; (4) group-dependent item parameters; (5) inclusion or exclusion of individual items; and (6) inter-item dependencies. An example of a test construction problem with practical constraints is presented. Using the three-parameter logistic model, an item bank of 1,000 items was drawn for the application of the test construction model, which was solved using the computer program LINPROG. Some alternative models of test construction are discussed. Three tables provide information about four solutions and list alternative objective functions in test construction

Dichotomous decisions based on dichotomously scored items: a case study

In a course in elementary statistics for psychology students using criterion-referenced achievement tests, the total test score, based on dichotomously scored items, was used for classifying students into those who passed and those who failed. The score on a test is considered as depending on a latent variable; it is assumed that the students can be dichotomized into the categories "mastery" (with scores on the latent variable above a cutting score), and "no mastery" (with scores below the cutting score on the latent variable). Two problems are considered: (a) How many students are classified incorrectly? Using the binomial error model a procedure is described for computing the classification proportions: p(mastery, passed), p(mastery, failed), p(no mastery, passed), and p(no mastery, failed), (b) What is the optimal cutting score on a test? Using a loss function a procedure for computing the optimal curring score is described