Reliability - the Precision of a Measurement (Version 2.0)

Reliability describes the precision of a measurement. This contribution begins by defining the concept of reliability and explaining why the reliability of a measurement is relevant. It then discusses the model assumptions that must be made in order to estimate the reliability of a measurement, and it presents five methods of estimating reliability: the test-retest method, the parallel-forms method, the split-half method, the internal consistency method, and the estimation of reliability using structural equation modelling. The contribution concludes with a brief outline of the commonalities and differences between classical test theory and the item response theory and the importance of these theories for the estimation of reliability

Measuring performance in dynamic decision making. Reliabiblity and validity of the Tailorshop simulation

The Tailorshop simulation is a computer-based dynamic decision-making task in which participants lead a fictional company for 12 simulated months. The present study investigated whether the performance measure in the Tailorshop simulation is reliable and valid. The participants were 158 employees from different companies. Structural equation models were used to test τ-equivalent measurement models. The results indicate that the trends of the company value between the second and the twelfth month are reliable variables. Furthermore, this measure predicted real-life job performance ratings by supervisors and was associated with the performance in another dynamic decision-making task. Thus, the trend of the company value provides a reliable and valid performance indicator for the Tailorshop simulation

The association between personality and cognitive ability: Going beyond simple effects

To examine the relationship between the Big Five and cognitive ability, we investigated whether we could replicate in a heterogeneous population sample the positive association between cognitive ability and Openness and Emotional Stability and its negative association with Conscientiousness. Besides analyzing the pure associations, we shed further light on sources of these associations by investigating potential moderating effects of education and labor force participation. Our results clearly replicate the previously found positive association between cognitive ability and Emotional Stability and Openness and the negative relationship between Conscientiousness and cognitive ability. The correlation between cognitive ability and Openness was found to be moderated by educational attainment, the negative association between Conscientiousness and cognitive ability was moderated by labor force participation

Cognitive ability beyond IQ

The present thesis investigates cognitive performance measures beyond IQ. I investigated the psychometric properties of implicit learning variables and dynamic decision making variables and their relation with general intelligence and professional success. The results suggest that dynamic decision making and implicit learning are substantially related with general intelligence and fit well into a hierarchical model of cognitive abilities. Furthermore, general intelligence is the best predictor for income, social status, and educational attainment. Dynamic decision making can predict supervisor ratings even beyond general intelligence

Reliabilität - die Genauigkeit einer Messung (Version 1.1)

Die Reliabilität beschreibt die Genauigkeit einer Messung. In diesem Beitrag wird das Konzept Reliabilität definiert und es wird erläutert, warum die Reliabilität einer Messung relevant ist. Danach wird diskutiert, welche Modellannahmen getroffen werden müssen, um die Reliabilität einer Messung zu schätzen und es werden fünf Methoden zum Schätzen der Reliabilität vorgestellt: die Re-Test Korrelation, die Parallel-Test Korrelation, die Split-Half Korrelation, die interne Konsistenz und das Schätzen der Reliabilität mit Strukturgleichungsmodellen. Abschließend wird in knapper Form auf Gemeinsamkeiten und Unterschiede der klassischen Testtheorie und der Item-Response Theorie und deren Bedeutung für die Schätzung der Reliabilität eingegangen

Big Five Personality Traits Predict Successful Transitions From School to Vocational Education and Training: A Large-Scale Study

Educational transitions play a pivotal role in shaping educational careers, and ultimately social inequality. Whereas parental socioeconomic status (SES) and cognitive ability have long been identified as key determinants of successful educational transitions, much less is known about the role of socio-emotional skills. To address this gap, the present study investigated whether Big Five personality traits predict success in the transition from secondary school to vocational education and training (VET) above and beyond SES, cognitive ability, and other covariates. Using data from Starting Cohort 4 of the German National Educational Panel Study (NEPS; N = 4,137), we defined seven indicators of successful transition: obtaining a VET position, number of acceptances for VET positions, starting a VET position, (the absence of) dropout intentions and actual dropout, final VET grade, and satisfaction with VET. The results revealed that some Big Five traits were incrementally associated with several indicators of transition success. Conscientiousness emerged as the single most relevant trait, predicting all the transition success indicators but 1 (dropout intentions). The other Big Five traits had much weaker and less consistent links with transition success. Extraversion predicted the final VET grade and obtaining a VET position; Agreeableness was linked to a higher risk of dropout. Openness and Emotional Stability had no incremental effects on transition success. There was also some evidence for both compensatory and synergistic interactive effects, with Openness moderating mainly the effects of parental SES (on dropout intentions, actual dropout, and number of acceptances), and Agreeableness moderating the effects of cognitive ability (on obtaining a VET position, number of acceptances, and satisfaction with VET). Although individual effect sizes were small, the Big Five’s joint contribution to transition success was non-negligible, and often larger than that of sociodemographic characteristics and cognitive ability. Our results suggest a hitherto underappreciated contribution of personality to successful transitions to VET

Don't Keep It Too Simple: Simplified Items Do Not Improve Measurement Quality

When formulating questionnaire items, generally accepted rules include: Keeping the wording as simple as possible and avoiding double-barreled items. However, the empirical basis for these rules is sparse. The present study aimed to systematically investigate in an experimental design whether simplifying items of a personality scale and avoiding double-barreled items (i.e., items that contain multiple stimuli) markedly increases psychometric quality. Specifically, we compared the original items of the Big Five Inventory-2 - most of which are either double-barreled or can be regarded as complexly formulated - with simplified versions of the items. We tested the two versions using a large, heterogeneous sample (N = 2,234). The simplified versions did not possess better psychometric quality than their original counterparts; rather, they showed weaker factorial validity. Regarding item characteristics, reliability, and criterion validity, no substantial differences were identified between the original and simplified versions. These findings were also replicated for the subsample of lower-educated respondents, who are considered more sensitive to complex item formulations. Our study thus suggests that simplifying item wording and avoiding double-barreled items in a personality inventory does not improve the quality of a questionnaire; rather, using simpler (and consequently more vague) item formulations may even decrease factorial validity

Is the emergence of functional ability decline in early old age related to change in speed of cognitive processing and also to change in personality?

To test whether the onset of functional ability decline in early old age is related to change in speed of cognitive processing and personality characteristics. Among 500 randomly sampled participants, the 230 cases that did not show impairment in functional ability were selected. Mean age at Time I was 62.4 years. For this subsample, the emergence of functional ability decline was tracked across a 12-year observation period. The emergence of functional ability decline was related to change in speed of cognitive processing. Decline in functional ability was also related to increased neuroticism and external control, whereas this was not the case regarding extraversión and internal control. Cognitive processing speed was shown to be a predictor of functional disability decline; in addition, the results provided initial evidence that functional ability decline in the early aging phase could be accompanied by changes in personality, particularly neuroticism and external control. (author's abstract

Recurrence extraction for functional programs through call-by-push-value

The main way of analyzing the complexity of a program is that of extracting and solving a recurrence that expresses its running time in terms of the size of its input. We develop a method that automatically extracts such recurrences from the syntax of higher-order recursive functional programs. The resulting recurrences, which are programs in a call-by-name language with recursion, explicitly compute the running time in terms of the size of the input. In order to achieve this in a uniform way that covers both call-by-name and call-by-value evaluation strategies, we use Call-by-Push-Value (CBPV) as an intermediate language. Finally, we use domain theory to develop a denotational cost semantics for the resulting recurrences.Comment: POPL 202