Subjective Indicators

Subjective indicators have been proven to possess predictive power for a large array of social and economic outcomes. However, most of these measures face serious psychometric shortcomings, namely that the items used are not psychometrically investigated. Further, for the assessment of one and the same construct various different item phrasing and response formats are used in different surveys. In the present paper several recommendations are made to increase the quality and by that also the acceptance and usage of subjective indicators. These include to develop more ultra-short but multi-item measures for subjective indicators. Further, surveys should try to use the same form of measurement, i.e. the same item phrasings and the same response scales. In terms of psychometric properties it is recommended to investigate reliability and validity of the indicators in as much depth as possible. In addition suggestions are made how to investigate the respondent’s judgmental process for the subjective indicators’ measures which allows to obtain a clearer picture of how the item is understood by the respondent and on which cues he bases his judgment.subjective indicators, reliability, validity, multi-item instruments, cognitive interviews

Subjective Indicators

"Subjective indicators have been proven to possess predictive power for a large array of social and economic outcomes. However, most of these measures face serious psychometric shortcomings, namely that the items used are not psychometrically investigated. Further, for the assessment of one and the same construct various different item phrasing and response formats are used in different surveys. In the present paper several recommendations are made to increase the quality and by that also the acceptance and usage of subjective indicators. These include to develop more ultra-short but multi-item measures for subjective indicators. Further, surveys should try to use the same form of measurement, i.e. the same item phrasings and the same response scales. In terms of psychometric properties it is recommended to investigate reliability and validity of the indicators in as much depth as possible. In addition suggestions are made how to investigate the respondent’s judgmental process for the subjective indicators’ measures which allows to obtain a clearer picture of how the item is understood by the respondent and on which cues he bases his judgment." [author's abstract

Kompetenzmessung in der Bildungsforschung. Zusammenfassung des Forums (7) der 5. Konferenz für Sozial- und Wirtschaftsdaten

In modernen Wissensgesellschaften ist Bildung die zentrale Voraussetzung für wirtschaftliches Wachstum und Wohlstand. Umgekehrt ist auch für das Individuum Bildung entscheidend für die erfolgreiche Teilhabe an der Gesellschaft und für die Bewältigung neuer Anforderungen in einer sich rasch wandelnden, globalisierten Welt. In den letzten Jahren basiert die Bildungsforschung zunehmend auf Daten umfangreicher und bevölkerungsrepräsentativer Umfragedaten, die somit Rückschlüsse auf die Verteilung der Bildung in den zugrundeliegenden Populationen erlauben. Um diese Verteilungen besser interpretieren und kontrastieren zu können, sind diese large-scale Bildungsforschungsstudien meist entweder in internationale Vergleichsstudien eingebettet, die somit einen kulturellen Vergleich ermöglichen,oder als Längsschnittstudien aufgesetzt mit dem Ziel, Veränderungen über Zeit und Auswirkungen von bildungspolitischen Weichenstellungen zu identifizieren.

Consistency of the Structural Properties of the BFI-10 Across 16 Samples From Eight Large-Scale Surveys in Germany

The assessment of the Big Five personality domains is standard practice in most large-scale social surveys nowadays. The instrument most widely used for this purpose is the BFI-10, an ultra-short measure assessing each Big Five domain with two items. Recent studies have identified issues with the structural properties of the BFI-10, especially its factorial validity. To investigate whether these issues arise from the instrument itself or biases due to translation or sampling, we examined the extent to which the structural properties of the BFI-10 in terms of descriptive statistics, intercorrelations, reliability, and factorial validity vary when keeping the target population and language constant. Results revealed that, across 16 independent samples (total N ~ 60,000) from eight large-scale surveys representative of the adult population in Germany, the structural properties of the BFI-10 were (a) largely consistent and (b) mostly adequate. Most importantly, in nearly all samples, patterns of loading were congruent with an idealized Big Five structure, thereby supporting factorial validity. These results demonstrate that the structural properties of the BFI-10 are highly stable and replicable in large-scale samples. Especially given its brevity, the BFI-10 can thus be regarded as adequate for use in large-scale survey settings

The association between personality and cognitive ability: Going beyond simple effects

To examine the relationship between the Big Five and cognitive ability, we investigated whether we could replicate in a heterogeneous population sample the positive association between cognitive ability and Openness and Emotional Stability and its negative association with Conscientiousness. Besides analyzing the pure associations, we shed further light on sources of these associations by investigating potential moderating effects of education and labor force participation. Our results clearly replicate the previously found positive association between cognitive ability and Emotional Stability and Openness and the negative relationship between Conscientiousness and cognitive ability. The correlation between cognitive ability and Openness was found to be moderated by educational attainment, the negative association between Conscientiousness and cognitive ability was moderated by labor force participation

PIAAC-L data collection 2016: technical report

This paper is part of a series of reports describing the technical implementation of PIAAC-L, the German PIAAC-Longitudinal project funded by the Federal Ministry of Education and Research. It is a follow-up to the technical reports for wave 1 (Zabal, Martin, & Rammstedt, 2016) and wave 2 (Zabal, Martin, & Rammstedt, 2017) and aims to describe the design, instruments, fieldwork processes, and data dissemination for wave 3

Beyond Competencies: Associations between Personality and School Grades Are Largely Independent of Subject-Specific and General Cognitive Competencies

The Big Five personality traits are established predictors of school grades. However, the mechanisms underlying these associations are not yet well understood. Effects of personality on grades might arise because behavioral tendencies facilitate learning and increase subject-specific competencies. Alternatively, personality effects on grades might be independent of cognitive competencies and reflect otherwise valued behaviors or teachers' grading practices. In the current study, we drew on large-scale data of 7th and 9th graders in Germany to explore the extent to which personality predicted grades even after accounting for competencies. Controlling for competencies and other key covariates, we cross-sectionally and longitudinally examined personality-grade associations across different school subjects, grade levels, and school types. Results indicate that the predictive power of personality is largely independent of subject-specific and general cognitive competencies. The largest effects emerged for conscientiousness. For openness, associations with grades partly overlapped with competencies, suggesting that openness may operate by fostering competencies. Overall, our results suggest that the associations between personality and grades unfold mostly independently of course mastery. This finding underlines the socioemotional value of personality in the classroom and encourages a more fine-grained view of the interplay between personality, competencies, classroom behavior, and grades

Does personality predict responses to the COVID-19 crisis? Evidence from a prospective large-scale study

The COVID-19 pandemic has significantly disrupted people's daily routines and infused their lives with considerable insecurity and uncertainty. However, individuals' responses to the pandemic vary widely. The present study investigates the role of personality traits for key aspects of people's responses to the COVID-19 crisis. In a prospective design using a large-scale panel study (N = 2217) that represents the heterogeneity of the adult population in Germany, we examined whether Big Five domains and facets measured prior to the pandemic predicted individuals' responses to the pandemic in terms of: (a) perceptions of infection risks; (b) behavioral changes to prevent infection; (c) beliefs in the effectiveness of policy measures to combat the further spread of coronavirus; and (d) trust in relevant policymakers and institutions regarding the handling of coronavirus. Results revealed that personality explained only a small portion (between 0.6% and 3.8%) of the variance in the four outcomes. Nonetheless, several Big Five domains and facets had at least small-to-moderate, and theoretically plausible, associations with the outcomes. Overall, Agreeableness and its Trust facet showed the most robust associations with the four outcomes. Most trait-outcome associations were also robust to controlling for three possible confounders (sex, age, and risk-group membership)

PIAAC-L data collection 2014: technical report; follow-up to PIAAC Germany 2012

The international PIAAC survey (Programme for the International Assessment of Adult Competencies) is part of a large-scale undertaking of the Organisation for Economic Cooperation and Development(OECD) to assess and monitor key adult competencies and hereby inform policy decisions. Countries participating in the first cycle of PIAAC conduct a large-scale assessment of literacy, numeracy, and problem solving in technology-rich environments1 together with the administration of a background questionnaire. This questionnaire covers numerous antecedents of skills as well as economic and social outcomes. National Project Managements for PIAAC are required to adhere to a strict and elaborate set of standards and guidelines in order to ensure a high quality of data and the comparability of the results. The 24 countries participating in the first round of PIAAC (including Germany) carried out their data collection in 2011/2012 (referred to as PIAAC 2012).2 The international results were made public in October 2013 (OECD, 2013; Rammstedt, 2013) and were taken up by a wide audience. The international Public Use Files are available at the OECD website

Measuring Growth Mindset: Validation of a Three-Item and a Single-Item Scale in Adolescents and Adults

A growth mindset is a belief that personal characteristics, specifically intellectual ability, are malleable and can be developed by investing time and effort. Numerous studies have investigated the associations between a growth mindset and academic achievement, and large intervention programs have been established to train adolescents to develop a stronger growth mindset. However, methodological research on the adequacy of the measures used to assess a growth mindset is scarce. In our study, we conducted one of the first comprehensive assessments of the psychometric properties of Dweck's widely used three-item Growth Mindset Scale in two samples (adolescents aged 14-19 years and adults aged 20-64 years). We test the comparability (i.e., measurement invariance) of the scale across these age groups. Furthermore, using the same two samples, we identified and validated a single-item measure to assess growth mindset in settings with severe time constraints. Results reveal that both the three-item and the single-item scales have acceptable psychometric properties regarding reliability, comparability, and validity. However, the results did not support some of the central tenets of mindset theory, such as that a growth mindset is positively linked to goal regulation and achievement, calling for future research on the criterion validity of a growth mindset