Computerized adaptive testing in industrial and organizational psychology

The overarching goal of this dissertation is to increase the precision and efficiency of the measurement tools that are used to make selection decisions in industrial/organizational psychology, by introducing psychometric innovations in the framework of computerized adaptive testing (CAT). Chapter 1 presents a general introduction to CAT and item response theory (IRT). Chapter 2 illustrates an automatic online calibration design that can be used in adaptive testing. The method makes it more attractive for respondents to participate during calibration, and increases the speed with which a CAT item bank can be calibrated. Chapter 3 demonstrates a straightforward method for conducting a test of measurement invariance and illustrates a method for modeling differential item functioning by assigning group-specific item parameters in the framework of IRT. Chapter 4 illustrates a method for verifying the results of an unproctored Internet test by using an extension of the stochastic curtailed truncated sequential probability ratio test (SCTSPRT). Simulation studies indicated that the SCTSPRT was almost four times shorter than a linear testing method while maintaining the same power of detection. Chapters 5 and 6 investigate the possibility of increasing the precision and shortening the test length of typical employment tests by efficiently administering and scoring items with multidimensional computerized adaptive testing (MCAT). Chapter 5 explores the possibility of using MCAT for administering and scoring the Adjustable Competence Evaluation; a computer adaptive cognitive ability test used in organizational selection. Chapter 6 explores the potential of administering and scoring items with MCAT for the NEO PI-R; a widely used personality test

A Multilevel Investigation of Resiliency Scales for Children and Adolescents: The Relationships Between Self-Perceived Emotion Regulation, Vagally Mediated Heart Rate Variability, and Personal Factors Associated With Resilience

Personal resiliency refers to individual attributes that are related to the process of successfully adapting to the environment in the face of adverse conditions, also known as resilience. Emotion regulation is increasingly found as a core component in mental health and found to modulate individual differences in the management of emotional responses. The Resiliency Scales for Children and Adolescents (RSCA; Prince-Embury, 2006, 2007) were designed to systematically identify and quantify core personal qualities of resiliency in youth, and includes Sense of Mastery scale (MAS), Sense of Relatedness scale (REL), and Emotional Reactivity (REA) scale. The following study was first conducted to confirm the Three-Factor model of Personal Resiliency in a Norwegian student sample using factor analytic procedures. Secondly and the main purpose of the study, was to investigate if personal resiliency is associated with self-reported measures related to emotion regulation, and with resting vagally mediated heart rate variability (vmHRV) as a psychophysiological index of emotion regulation capacity. A revised scale adapted to the Norwegian sample was developed. Results indicate that protective indices related to personal resiliency are associated with both self-reported adaptive emotion regulation and outcome, and partly related to high capacity for emotion regulation indicated by vmHRV. Risk related to personal vulnerability was associated with maladaptive emotion regulation and outcome, but was not associated with emotion regulation capacity. Together the findings provide supporting evidence of both self-reported and psychophysiological correlates between emotion regulatory processes and personal resiliency indicated by RSCA

Critical Values for Yen’s Q3: Identification of Local Dependence in the Rasch model using Residual Correlations

The assumption of local independence is central to all IRT models. Violations can lead to inflated estimates of reliability and problems with construct validity. For the most widely used fit statistic Q3 there are currently no well-documented suggestions of the critical values which should be used to indicate local dependence, and for this reason a variety of arbitrary rules of thumb are used. In this study, we used an empirical data example and Monte Carlo simulation to investigate the different factors that can influence the null distribution of residual correlations, with the objective of proposing guidelines that researchers and practitioners can follow when making decisions about local dependence during scale development and validation. We propose that a parametric bootstrapping procedure should be implemented in each separate situation in order to obtain the critical value of local dependence applicable to the data set, and provide example critical values for a number of data structure situations. The results show that for the Q3 fit statistic no single critical value is appropriate for all situations, as the percentiles in the empirical null distribution are influenced by the number of items, the sample size, and the number of response categories. Furthermore, our results show that local dependence should be considered relative to the average observed residual correlation, rather than to a uniform value, as this results in more stable percentiles for the null distribution of an adjusted fit statistic

A shortened version of Raven's standard progressive matrices for children and adolescents

Numerous developmental studies assess general cognitive ability, not as the primary variable of interest, but rather as a background variable. Raven's Progressive Matrices is an easy to administer non-verbal test that is widely used to measure general cognitive ability. However, the relatively long administration time (up to 45 min) is still a drawback for developmental studies as it often leaves little time to assess the primary variable of interest. Therefore, we used a machine learning approach - regularized regression in combination with cross-validation - to develop a short 15-item version. We did so for two age groups, namely 9 to 12 years and 13 to 16 years. The short versions predicted the scores on the standard full 60-item versions to a very high degree r = 0.89 (9-12 years) and r = 0.93 (13-16 years). We, therefore, recommend using the short version to measure general cognitive ability as a background variable in developmental studies

Virtual reality as a recovering environment - Implications for design principles

Peer reviewe