Dichotomous decisions based on dichotomously scored items: a case study

In a course in elementary statistics for psychology students using criterion-referenced achievement tests, the total test score, based on dichotomously scored items, was used for classifying students into those who passed and those who failed. The score on a test is considered as depending on a latent variable; it is assumed that the students can be dichotomized into the categories "mastery" (with scores on the latent variable above a cutting score), and "no mastery" (with scores below the cutting score on the latent variable). Two problems are considered: (a) How many students are classified incorrectly? Using the binomial error model a procedure is described for computing the classification proportions: p(mastery, passed), p(mastery, failed), p(no mastery, passed), and p(no mastery, failed), (b) What is the optimal cutting score on a test? Using a loss function a procedure for computing the optimal curring score is described

Coefficients for tests from a decision theoretic point of view

From a decision theoretic point of view a general coefficient for tests, d, is derived. The coefficient is applied to three kinds of decision situations. First, the situation is considered in which a true score is estimated by a function of the observed score of a subject on a test (point estimation). Using the squared error loss function and Kelley’s formula for estimating the true score, it is shown that d equals the reliability coefficient from classical test theory. Second, the situation is considered in which the observed scores are split into more than two categories and different decisions are made for the categories (multiple decision). The general form of the coefficient is derived, and two loss functions suited to multiple decision situations are described. It is shown that for the loss function specifying constant losses for the various combinations of categories on the true and on the observed scores, the coefficient can be computed under the assumptions of the beta-binomial model. Third, the situation is considered in which the observed scores are split into only two categories and different decisions are made for each category (dichotomous decisions). Using a loss function that specifies constant losses for combinations of categories on the true and observed score and the assumption of an increasing regression function of t on x, it is shown that coefficient d equals Loevinger’s coefficient H between true and observed scores. The coefficient can be computed under the assumption of the beta-binomial model. Finally, it is shown that for a linear loss function and Kelley’s formula for the regression of the true score on the observed score, the coefficient equals the reliability coefficient of classical test theory

The concept of quality of life in dementia in the different stages of the disease

In order to conceptually define quality of life (QOL) in dementia, the literature on QOL in the elderly population, in chronic disease and in dementia was studied. Dementia is a progressive, age-related, chronic condition and to avoid omissions within the dementia-specific concept of QOL, a broad orientation was the preferred approach in this literature study. Adaptation is a major outcome in studies investigating interventions aimed at improving QOL in chronic conditions, but to date, it has not been used in the definition of QOL. It is argued that adaptation is an important indication of QOL in people with chronic diseases and therefore also in dementia. Some crucial issues in assessing dementia-related QOL that are relevant to clarify the continuing debate on whether QOL, particularly in dementia, can be measured at all, are discussed. Then the following conceptual definition is offered: dementia-specific QOL is the multidimensional evaluation of the person-environment system of the individual, in terms of adaptation to the perceived consequences of the dementi

CAT for Personality Items Zeitschrift für Psychologie

Abstract. A computerized adaptive testing (CAT) procedure was simulated with ordinal polytomous personality data collected using a conventional paper-and-pencil testing format. An adapted Dutch version of the dominance scale of Gough and Heilbrun's Adjective Check List (ACL) was used. This version contained Likert response scales with five categories. Item parameters were estimated using Samejima's graded response model from the responses of 1,925 subjects. The CAT procedure was simulated using the responses of 1,517 other subjects. The value of the required standard error in the stopping rule of the CAT was manipulated. The relationship between CAT latent trait estimates and estimates based on all dominance items was studied. Additionally, the pattern of relationships between the CAT latent trait estimates and the other ACL scales was compared to that between latent trait estimates based on the entire item pool and the other ACL scales. The CAT procedure resulted in latent trait estimates qualitatively equivalent to latent trait estimates based on all items, while a substantial reduction of the number of used items could be realized (at the stopping rule of 0.4 about 33% of the 36 items was used)

Network Psychometrics

This chapter provides a general introduction of network modeling in psychometrics. The chapter starts with an introduction to the statistical model formulation of pairwise Markov random fields (PMRF), followed by an introduction of the PMRF suitable for binary data: the Ising model. The Ising model is a model used in ferromagnetism to explain phase transitions in a field of particles. Following the description of the Ising model in statistical physics, the chapter continues to show that the Ising model is closely related to models used in psychometrics. The Ising model can be shown to be equivalent to certain kinds of logistic regression models, loglinear models and multi-dimensional item response theory (MIRT) models. The equivalence between the Ising model and the MIRT model puts standard psychometrics in a new light and leads to a strikingly different interpretation of well-known latent variable models. The chapter gives an overview of methods that can be used to estimate the Ising model, and concludes with a discussion on the interpretation of latent variables given the equivalence between the Ising model and MIRT.Comment: In Irwing, P., Hughes, D., and Booth, T. (2018). The Wiley Handbook of Psychometric Testing, 2 Volume Set: A Multidisciplinary Reference on Survey, Scale and Test Development. New York: Wile

Physical-activity support for people with intellectual disabilities:development of a tool to measure behavioural determinants in direct support professionals

Background Physical-activity approaches for people with intellectual disabilities (ID) are more likely to be effective and sustainable if they also target direct support professionals' behaviour. However, no tools to measure the behavioural determinants for direct support professionals are available as of yet. This study aims to construct a self-report tool to measure direct support professionals' behavioural determinants in physical-activity support for people with ID and to analyse its psychometric properties. Methods The tools' sub-scales and items corresponded with a proposed conceptual model. A pilot study was carried out to investigate and improve content validity. Construct validity and measurement precision were examined using item response theory models with data from a convenience sample of 247 direct support professionals in the support of people with ID. Results Results supported the three theory-driven behaviour scales and indicated reasonable to good construct validity. The marginal reliability for the scales ranged from 0.84 to 0.87, and adequate measurement precision along the latent continua was found. Conclusions The tool appears to be promising for measuring the behavioural determinants of direct support professionals for the physical-activity support of people with ID and has potential as a tool for identifying areas to focus on for interventions and policies in the future

Phenotypic Complexity, Measurement Bias, and Poor Phenotypic Resolution Contribute to the Missing Heritability Problem in Genetic Association Studies

Background The variance explained by genetic variants as identified in (genome-wide) genetic association studies is typically small compared to family-based heritability estimates. Explanations of this ‘missing heritability’ have been mainly genetic, such as genetic heterogeneity and complex (epi-)genetic mechanisms. Methodology We used comprehensive simulation studies to show that three phenotypic measurement issues also provide viable explanations of the missing heritability: phenotypic complexity, measurement bias, and phenotypic resolution. We identify the circumstances in which the use of phenotypic sum-scores and the presence of measurement bias lower the power to detect genetic variants. In addition, we show how the differential resolution of psychometric instruments (i.e., whether the instrument includes items that resolve individual differences in the normal range or in the clinical range of a phenotype) affects the power to detect genetic variants. Conclusion We conclude that careful phenotypic data modelling can improve the genetic signal, and thus the statistical power to identify genetic variants by 20-99