A meeting report: cross-cultural comparability of questionnaire measures in large-scale international surveys

The value of cross-country comparisons is at the heart of large-scale international surveys. Yet the validity of such comparisons is often challenged, particularly in the case of latent traits whose estimates are based on self-reported answers to a small number of questionnaire items. Many believe self-reports to be unreliable and not comparable, and indeed, formal statistical procedures very often reject the assumption that the questions are understood and answered in the same way in different countries (measurement invariance). A methodological conference on the comparability of questionnaire scales was hosted by the OECD on 8 and 9 November 2018. This meeting report summarises the discussions held at the conference about measurement invariance testing and instrument design. The report first provides a brief introduction to the measurement models and the accompanying invariance analyses typically used in the industry of large-scale international surveys and points to the main limitations of these current standard approaches. It then presents classical and novel ways to deal with imperfect comparability of measurements when scaling and reporting on continuous traits and on categorical latent variables. It finally discusses the extent to which item design can improve the cross-country comparability of the measured constructs (e.g. by adopting innovative item formats such as anchoring vignettes and situational judgement test items). It concludes with some general considerations for survey design and reporting on invariance analyses and survey results

Analysis of qualitative data: introductory topics

Analysis of Qualitative Dat

Analysis of qualitative data: new developments

Analysis of Qualitative Dat

A note on the conditional moments of a multivariate normal distribution confined to a convex set

Let Y be an N([mu], [Sigma]) random variable on Rm, 1 = 0, let [beta]s(v) be the expected value of (v, Y) - (v, [mu])s and let [gamma]s(v) be the conditional expected value of (v, Y) - (v, [mu]c)s given Y [set membership, variant] C. For s >= 1, [gamma]s(v)Normal distributions moments convex sets inequalities

How Often Is the Misfit of Item Response Theory Models Practically Significant?

Educational Measurement: Issues and Practice, Vol. 33, No. 1, pp. 23-35Standard 3.9 of the Standards for Educational and Psychological Testing (1999) demands evidence of model fit when item response theory (IRT) models are employed to data from tests. Hambleton and Han (2005) and Sinharay (2005) recommended the assessment of practical significance of misfit of IRT models, but few examples of such assessment can be found in the literature concerning IRT model fit. In this article, practical significance of misfit of IRT models was assessed using data from several tests that employ IRT models to report scores. The IRT model did not fit any data set considered in this article. However, the extent of practical significance of misfit varied over the data sets