The purpose of this paper is to link empirical Bayes methods with two specific topics in item response theory--item/subtest regression, and testing the goodness of fit of the Rasch model--under the assumptions of local independence and sufficiency. It is shown that item/subtest regression results in empirical Bayes estimates only if the Rasch model holds. Additionally, it is shown that a newly-derived exploratory goodness-of-fit test for the Rasch model, which does not need item and person parameter estimates, can be seen as an empirical Bayes test. This test compares the observed proportions of correct answers to one specific item, given any pattern that leads to a number-right score

Engelen, Ronald J.H.

Jannarone, Robert J.

English

University of Twente Research Information

DOCUMENT RESUMEED 322 199 TM 015 394AUTHOR Engelen, Ronald J. H.; Jannarone, Robert J.TITLE A Connection between Item/Subtest Regression and theRasch Model. Research Report 89-1.INSTITUTION Twente Univ., Enschede (Netherlands). Dept. ofEducation.PUB DATE Dec 89NOTE 17p.AVAILABLE FROM Bibliot.heek, Department of Education, University ofTwente, P.O. Box 217, 7500 AE Enschede, TheNetherlands.PUB TiPE Reports - Evaluative/Feasibility (142)EDRS PRICE MF01/PC01 Plus Postage.DESCRIPTORS *Bayesian Statistics; Comparative Analysis; Equations(Mathematics); *Estimation (Mathematics); *Goodnessof Fit; Item Analysis; Item Response Theory;*Mathematical Models; *Regression (Statistics); TestItemsIDENTIFIERS *Item Subtest Regression; *Rasch ModelABSTRACTThe purpose of this paper is to link empirical Bayesmethods with two specific topics in item responsetheory--item/subtest regression, and testing the goodness of fit ofthe Rasch model--under the assumptions of local independence andsufficiency. It is shown that item/subtest regression results inempirical Bayes estimates only if the Rasch model holds.Additionally, it is shown that a newly-derived exploratorygoodness-of-fit test for the Rasch model, which does not need itemand person parameter estimates, can be seen as an empirical Bayestest. This test compares the observed proportions of correct answersto ont specific item, given any pattern that leads to a number-rightscore. These proportions should be a: lilt equal. (RLC)Reproductions supplied by EDRS are the best that can be madefrom the original document.A Connection Between Item /SubtestRegression and the Rasch Model ResearchReport89-1U S DEPARTMENT Of EDUCATIONOffice of Ecucational Reseerch and ImprovementEDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)11)0<iis document has been reproducer] asreceived from the !Ninon or organizationonginsting it.0 Minor Changes have been made to imprsyereproduction qualityPoint sofvaworopinionsstetedutthadocument do not necessarily represent officialOERI position or pOliCy1Ronald J.H. EngelenRobert J. Jannarone"PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BYE.1 SSENTO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC"BEST COPY AVAILABLEUniversity of TColofon:Typing: L.A.M. Bosch-PadbergCover design: Audiovisuele Sectie TOLAB ToegepasteOnderwijskundePrinted by: Centrale Reproductie- afdeling0JA Connection Between Item-Subtest Regressionand the Rasch ModelRonald J.H. EngelenBob J. Jannarone(A connection between item-subtest regression and the Raschmodel / Ronald J.H. Engelen and Robert J. Jannarone -Enschede : University of Twente, Department of Education,December, 1989. - 9 pages5Item-Subtest Regression1AbstractEmpirical Bayes methods are linked with item re-oonse theoryunder the assumptions of local ind,.pendence and sufficiency.It is shown that item-subtest regression results in empiricalBayes estimates if and only if the Rasch model holds.Additionally, it will be shown that a newly derived,exploratory, Rasch model test can be seen as an empiricalBayes test.Key- words: Empirical Bayes estimation, Rasch model, itemsubtest regression, sufficiency.sItem-Subtest Regression2IntroductionThe purpose of this paper is to link empirical Bayes methodswith two specific topics in item response theory (IRT): item-subtest regression and testing the goodness of fit of theRasch model. It will be shown that item-subtest regressionresults in empirical Bayes estimates if and only if the Raschmodel ( Rasch, 1980) holds. Additionally, it will be shownthat a Rasch model (RM) test can be derived as an empiricalBayes test from first principles.To briefly introduce empirical Bayes estimation, let usassume that an observation z of a random variable Z is made,where the distribution function of Z depends on a parameter0, i.e., the distribution function of Z is given by F(zI0).In empirical Bayes estimation, just as in "classical" Bayesestimation, the assumption of a prior distribution functionG(0) for the unknown parameter 0 is essential. In addition,however, "previous" data is used to obtain a (in some way)reasonable estimator of G(0), and hence of 0. In some casesit is possible to obtain an empirical Bayes estimator of 9without explicit estimation of G(0). For a more detailedintroduction to empirical Bayes methods, see, for instance,Robbins (1955, 1964) or Maritz (1970).Sample item-subtest regressions, which are used by Lord(1980) to construct estimators for item-response functions,are obtained by computing the proportion of examinees thatanswered one specific item correctly, given their number-correct scores on the test minus the item concerned.fiItem-Subtest Regression3Empirical Bayes Methods in IRT ModelsAs an introduction to the concept of empirical Bayesestimation in IRT models, we will first explain how theconcept works in the simplest 1RT model, the binomial model.In this model, all K items have the same itemcharacteristic curve (ICC), i.e.T(8) 1)(X6=118) = e for allm = 1,...,K, where 8 is the (latent) ability. So, the totalscore Y is binomially distributed with parameters K and 8.The Bayes point estimator 8G(y) of the mean ICC, which isdenoted by T(0) i!1Im(8)/K, under squared error loss isgiven by(1)I etyy;8)dG(G)8G(Y)pK(y; 0) dG (e)where pK(y;8) is the density or discrete probability functionof y given e, and G(e) the prior distribution function of 8(Lehmann, 1983). Note that T(8) = e. For the binomial case(1) leads to(2)(y +1) nK+1(1".1)8G(Y)(K+1) pK(y)where pK +1(z) denotes the probability of obtaining a totalscore of z on a (K+1)-item test (Meredith and Kearns, 1973).8Item-Subtest Regression4Now suppose that a random sample of examinees of size Nis available. The empirical Bayes estimate of 0 is thenobtained by simply substituting observed frequencies fortheoretical probabilities. Since scores on a (K+1)-item testcan not be observed however, (2) must be modified in such away that only observable quantities remain. Apart from sometechnical details, this is done by computing empirical Bayesestimates on the subtests in which the m-th (m=1,...,K) itemhas been deleted, and evaluating the mean of these estimatesto get a more stable result. It can be shown that theseestimates are consistent and that the empirical Bayes riskconverges to the Bayes risk. More details zan be found inCressie (1982), Jannarone (1979), Meredith and Kearns (1973),Kearns and Meredith (1975) and Robbins (1955).It should be mentioned that some sort of smoothing ofempirical Bayes estimates may be useful, such as constrainingthe estimates to increase mon 'tonically in the total score y(van Houwelingen, 1977; Jannarone, 1979).With the binomial model as a guide-line, it is now easyto evaluate empirical Bayes estimates for the ICC's forgeneral IRT models, without making any assumptions whatsoeveron the form of the ICC (i.e., non-parametrically).We want to compute the empirical Bayes estimate of them-th ICC, tm(0). As in the binomial model, this will done byusing the responses on subtest S(m) containing all but them-th item. In contrast with the binomial model, where allitems are equivalent, this estimate has to be computed foreach subtest response vector separately. Hence, the empiricalItem-Subtest Regression5Bayes estimate will be based on x(m), denoting the responsevector on S(m). In the following we will use subscripts (m)to denote operations were the m-th item is excluded.We will need the following Assumption:(3) P (Xm 1, X (m) = x (m) le) = P (Xel le) P (X (m) =x (m) le),for all m and all vectors x(m). For that matter, it can beeasily shown that (3) is equivalent with the assumption oflocal independence.With (3) and the above notations, the Bayes estimate of'Cm (e) based on x(m) is defined by(4) 8(m) ) = I (Tra(e) IX =xRewriting (4) leads to(5)I Tm(e)P (X(m) = x (m) I dG (8)8(m) (x I P (m) x (m) I e)dG (0)Now suppose that(6) P (X (m) (m) qcs(x(m));epthc(m))for some functions q and h and for all x(m), or equivalentlysuppose that the sufficient factorization criterion holds onthe subtest S(m). Then10(7) 8(m)(x(m))Item-Subtest RegressionI I rt(0)q(s (x (n)); 0) dGJ cs (x (n) ) ;0) dG (0)8(m) (y(m) )6provided that s(x(m)) = s(y(m)). Hence, in this case, theBayes estimates for tm(8) based on two response patterns x(m)and y(m) are equal if the sufficient statistic for thesepatterns are equal. But we have sufficiency of the item-deleted test score in the RM as well as in the two-parameterlogistic model with known (not necessary equal)discrimination parameters only. So, for the RM, the Bayesestimate of the m-th ICC can now be based on the sufficientstatistic R(m) -non Xn. This estimate is easily obtained bysubstituting r(m) for x(m) in formula (4), which leads to(8) 8(m)(x(m)Ir(n)) =P (Xel, R (m) imr (m) )P (m) =r (m) )which is the usual item true-score regression (Lord, 1980,p. 251-252).The empirical Bayes estimate of Tim(8) based on R(m) isnow obtained by substituting observed proportions fortheoretical probabilities, i.e.,AAP (Xm=1, R (m) (m))(9) 8(m)(x(m)Ir (m)) =P (R(m)r7r(m))11Item- Subtest Regression7Note that our results point towards a possible goodness-of-fit test for the m-th item, given that the RM holds forthe subtest S(m): If the observed proportion of examineesanswering the m-th item rorrectly for a subtest score isroughly equal for each pattern that leads to the subtestscore, then the RM should also hold for the m-th item,otherwise it should not. Such a test would be explorative, ina way that resembles the 'splitter' technique proposed byStelzl (1979) and also described by Molenaar (1983).DiscussionIt has been shown that an empirical Bayes justification forusing item-subtest regressions as item response functionestimates holds only for the Rasch models. Indeed, (4) and(6) show that item-subtest regression is inappropriateotherwise. Similar conclusions may hold regarding the use ofnumber-correct scores in other IRT models as well.Additionally, an (exploratory) goodness-of-fit test forthe Rasch model, which does not need item and personparameter estimates has been indicated. This test comparesthe observed proportions of correct answers to one specificitem, given any pattern that leads to a number-right scoreof, say, R' on the remaining items. These proportions shouldbe about equal.12Item-Subtest Regression8ReferencesCressie, N. (1982). A useful empirical Bayes identity. Anaalsof Statistics, 19., 625-629.Jannarone, B. (1979). Emoirical Bayes estimates of expected(true) scores for two latent trait models. UnpublishedPh.D. Thesis: University of California, Berkeley.Kearns, J., & Meredith, W. (1975). Methods for evaluatingempirical Bayes estimates of latent trait scores.ptyrhomatz2ka, AZ 373-394.Molenaar, I.W. (1983) Sume improved diagnostics for failureof the Rasch model. Psychometrika, AI, 49-72.Lehmann, E.L. (1983). Theory of point estimation. New York:Wiley.Lord, P.M. (1980). ApplicAtionugitenusapansethitoratagracticALleatinajarn.ohlEms. Hillsdale: Lawrence Erlbaum.Maritz, J.S. (1970). Empirical Bayes methods. London:Metheun.Meredith, W., & Kearns, J. (1973). Empirical Bayes pointestimates of latent trait scores without knowledge ofthe trait distribution Psvchometrika, 21, 533-5E4.Rasch, G. (1980). ErsthabiliaticmacialsfarsomeintaligimalAncLAttainmenttfuti,. Chicago: The University of ChicagoPress.Robbins, H. (1955). AD empirical Bstatistics. In J. Nsyman (Ed.): Proceedings of the ThirdBerkeley symposium on mathematical statistics andprobability, Volume 1. Berkeley: U.C. Press.1_3Item-Subtest Regression9Robbins, H. (1964). The empirical Bayes approach tostatistical decision problems. Annals of MatStatistics, 25, 1-20.Stelzl, I. (1979). Ist der Modelltest des Rasch-Modellsgeelgnet, Homogenitftshypothesen zu prOfen? Ein berichtOber Simulationsstudien mit inhomogenen Daten. [Is theRasch model sensitive to tests of homogenity?]Zeitschrift Experimentelle and AngewandteMicialazia, 21, 652-672.Van Houwelingen, J.C. (1977). Monotizing empirical Hayesestimators for a class of discrete distributions withmonotone likelihood ratio. Statistics Neerlandica, 21,95-104.14IlnizazaitaAL2mantm.2amrhada..The Netherlands.RR-89-1 R.J.H. Engelen i R.J. Jannarone, A connection betweenitem/subtest regression and the Mach modelRR-88-18 H.J. Vos, Applications of decision theory to computer basedadaptive instructional systemsRR-88-17 H. Kelderman, Loglinear multidimensional IRT models forpolytomously scored itemsRR-88-16 H. Kelderman, An IRT model for item responses that aresubject to omission and/or intrusion errorsRR-88-15 H.J. Vos, Simultaneous optimisation of decisions using alinear utility functionRR-88-14 J.J. Adams, The construction of two-stage testsRR-88-13 J. Kogut, Asymptotic distribution of an IRT person fit indexRR-88-12 B. van der Burg i G. Dijksterhuis, Nonlinear canonicalcorrelation analysis of multiway dataRR-88-11 D.L. Mug i m.p.r. Berger, Empirical comparison betweenfactor analysis and item response modelsRR-88-10 H. Kelderman i G. /Unready, Loglinear-latent-class models fnrdetecting item biasRR-88-9 N.J. van der Linden i T.J.H.N Eggen, The Ranch model as amodel for paired comparisons with an indivldual tie parameterRR-08-8 R.J.H. Engelen, N.J. van der Linden, i S.J. Oosterloo, Iteminformation in the Basch modalRR-88-7 J.H.A.N. Bikers, Towards an anthoring system for itemconstructionRR-88-6 H.J. Vos, The use of decision theory in the MinnesotaAdaptive Instructional SystemRR-88-5 N.J. van der Linden, Optimizing incomplete sample designs foritem response model parametersRR-88-4 J.J. Adams, A note on solving large-scale zero-onepro?ramming problems15RR-88-3 E. E'ekkooi-Timminga, A cluster-based method for testconstructionRR-88-2 N.J. van der Linden i J.J. Adema, Algorithmic test designusing classical item parametersRR-88-1 E. van der Burg i J. de Loom', Nonlinear redundancy analysisJasesersajlepsuLte can be obtained at coats from Bibliotheek,Department of Education, University of Twente, P.O. Box 217,7500 AZ Enschede, The Netherlands.IItrnent ofCATION. .ation byDepartment of Educationof nte

A connection between item/subtest regression and the Rasch model

Abstract

Similar works

Full text

Available Versions

University of Twente Research Information

NARCIS