Subjective Indicators

Subjective indicators have been proven to possess predictive power for a large array of social and economic outcomes. However, most of these measures face serious psychometric shortcomings, namely that the items used are not psychometrically investigated. Further, for the assessment of one and the same construct various different item phrasing and response formats are used in different surveys. In the present paper several recommendations are made to increase the quality and by that also the acceptance and usage of subjective indicators. These include to develop more ultra-short but multi-item measures for subjective indicators. Further, surveys should try to use the same form of measurement, i.e. the same item phrasings and the same response scales. In terms of psychometric properties it is recommended to investigate reliability and validity of the indicators in as much depth as possible. In addition suggestions are made how to investigate the respondent’s judgmental process for the subjective indicators’ measures which allows to obtain a clearer picture of how the item is understood by the respondent and on which cues he bases his judgment.subjective indicators, reliability, validity, multi-item instruments, cognitive interviews

Subjective Indicators

"Subjective indicators have been proven to possess predictive power for a large array of social and economic outcomes. However, most of these measures face serious psychometric shortcomings, namely that the items used are not psychometrically investigated. Further, for the assessment of one and the same construct various different item phrasing and response formats are used in different surveys. In the present paper several recommendations are made to increase the quality and by that also the acceptance and usage of subjective indicators. These include to develop more ultra-short but multi-item measures for subjective indicators. Further, surveys should try to use the same form of measurement, i.e. the same item phrasings and the same response scales. In terms of psychometric properties it is recommended to investigate reliability and validity of the indicators in as much depth as possible. In addition suggestions are made how to investigate the respondent’s judgmental process for the subjective indicators’ measures which allows to obtain a clearer picture of how the item is understood by the respondent and on which cues he bases his judgment." [author's abstract

Kompetenzmessung in der Bildungsforschung. Zusammenfassung des Forums (7) der 5. Konferenz für Sozial- und Wirtschaftsdaten

In modernen Wissensgesellschaften ist Bildung die zentrale Voraussetzung für wirtschaftliches Wachstum und Wohlstand. Umgekehrt ist auch für das Individuum Bildung entscheidend für die erfolgreiche Teilhabe an der Gesellschaft und für die Bewältigung neuer Anforderungen in einer sich rasch wandelnden, globalisierten Welt. In den letzten Jahren basiert die Bildungsforschung zunehmend auf Daten umfangreicher und bevölkerungsrepräsentativer Umfragedaten, die somit Rückschlüsse auf die Verteilung der Bildung in den zugrundeliegenden Populationen erlauben. Um diese Verteilungen besser interpretieren und kontrastieren zu können, sind diese large-scale Bildungsforschungsstudien meist entweder in internationale Vergleichsstudien eingebettet, die somit einen kulturellen Vergleich ermöglichen,oder als Längsschnittstudien aufgesetzt mit dem Ziel, Veränderungen über Zeit und Auswirkungen von bildungspolitischen Weichenstellungen zu identifizieren.

Zur Bestimmung der Güte von Multi-Item-Skalen: eine Einführung

'Um interessierende Merkmale zu erfassen, werden häufig Fragebogen (ob persönlichmündlich oder telefonisch in Form eines standardisierten Interviews, schriftlich oder online vorgegeben) eingesetzt. Im Zuge der Entwicklung oder der Auswahl eines solchen Fragebogen stellt sich primär die Frage, wie gut dieser Fragebogen für den Untersuchungszweck geeignet ist. Ziel und Zweck dieser Einführung ist, den Blick für die Güte von Fragebogen zu schärfen sowie Verfahren zur Güteüberprüfung zu vermitteln. So sind Güteüberlegungen entscheidend in Situationen, in denen für eine bestimmte Untersuchung ein geeignetes Instrument ausgewählt werden soll. Auch bei der selbständigen Fragebogenentwicklung ist es entscheidend, dessen Qualität zu überprüfen. Hier werden Multi-Item-Skalen betrachtet, also die Teile eines Fragebogens oder gesamte Fragebogen, in denen ein Konstrukt mittels mehrerer Items erfasst wird, deren Beantwortung dann gemittelt oder aufsummiert wird. Von zentraler Bedeutung für die Beurteilung der Qualität von Multi-Item-Skalen sind die so genannten Hauptgütekriterien, nämlich die Objektivität, die Reliabilität und die Validität des Verfahrens. Jedes der drei Kriterien lässt sich in drei oder vier Aspekte untergliedern, die im Folgenden näher dargestellt werden sollen'. (Autorenreferat)'The present paper gives an introduction how to assess the quality of a multi-item scale. The three main criteria, objectivity, reliability, and validity are presented as well as possibilities for their empirical examination.' (author's abstract)

Welche Vorhersagekraft hat die individuelle Persönlichkeit für inhaltliche sozialwissenschaftliche Variablen?

'In der vorliegenden Studie wurde überprüft, in wie weit die individuelle Persönlichkeit in Form der fünf grundlegenden Persönlichkeitsdimensionen die Vorhersage inhaltlicher sozialwissenschaftlicher Variablen verbessern kann. Verbessern wurde in diesem Zusammenhang als eine zusätzliche Varianzerklärung zu der der klassischen soziodemographischen Variablen Alter, Geschlecht und Bildung verstanden. Als Datenbasis diente der kombinierte Datensatz des ALLBUS 2004 und des ISSP 2003/2004, deren zentrale Themenkomplexe als abhängige Variablen untersucht wurden. Die Ergebnisse zeigen, dass für sämtliche Themenkomplexe eine oder mehrere Persönlichkeitsvariablen deutlich zur Verbesserung der Vorhersage beitrugen. Welche Persönlichkeitsdimension die Vorhersage im einzelnen verbesserte sowie das Ausmaß dieser Verbesserung variierte stark zwischen den einzelnen Themenkomplexen. Es kann somit geschlussfolgert werden, dass die standardmäßige Erfassung der Persönlichkeit die prädiktive Validität sozialwissenschaftlicher Umfragen deutlich erhöhen könnte.' (Autorenreferat)'This study examines in how far the individual's personality, conceptualized as the five most basic dimensions of personality, can improve predicting content variables in the social sciences. Improvement here is defined as additional variance that can be explained in addition to the standard socio-demographic variables age, gender, and education. The ALLBUS 2004 and the ISSP 2003/2004, whose central module topics were investigated as dependent variables, were combined to serve as data basis. Results show that, for all topics, one or more personality variables could significantly improve the prediction. Which dimension of personality leads to the improvement and the amount of variance explained by personality vary greatly between topics. It can be concluded that routinely assessing personality has the potential to significantly increase the predictive validity in social science survey research.' (author's abstract)

Consistency of the Structural Properties of the BFI-10 Across 16 Samples From Eight Large-Scale Surveys in Germany

The assessment of the Big Five personality domains is standard practice in most large-scale social surveys nowadays. The instrument most widely used for this purpose is the BFI-10, an ultra-short measure assessing each Big Five domain with two items. Recent studies have identified issues with the structural properties of the BFI-10, especially its factorial validity. To investigate whether these issues arise from the instrument itself or biases due to translation or sampling, we examined the extent to which the structural properties of the BFI-10 in terms of descriptive statistics, intercorrelations, reliability, and factorial validity vary when keeping the target population and language constant. Results revealed that, across 16 independent samples (total N ~ 60,000) from eight large-scale surveys representative of the adult population in Germany, the structural properties of the BFI-10 were (a) largely consistent and (b) mostly adequate. Most importantly, in nearly all samples, patterns of loading were congruent with an idealized Big Five structure, thereby supporting factorial validity. These results demonstrate that the structural properties of the BFI-10 are highly stable and replicable in large-scale samples. Especially given its brevity, the BFI-10 can thus be regarded as adequate for use in large-scale survey settings

PIAAC-L data collection 2016: technical report

This paper is part of a series of reports describing the technical implementation of PIAAC-L, the German PIAAC-Longitudinal project funded by the Federal Ministry of Education and Research. It is a follow-up to the technical reports for wave 1 (Zabal, Martin, & Rammstedt, 2016) and wave 2 (Zabal, Martin, & Rammstedt, 2017) and aims to describe the design, instruments, fieldwork processes, and data dissemination for wave 3

The association between personality and cognitive ability: Going beyond simple effects

To examine the relationship between the Big Five and cognitive ability, we investigated whether we could replicate in a heterogeneous population sample the positive association between cognitive ability and Openness and Emotional Stability and its negative association with Conscientiousness. Besides analyzing the pure associations, we shed further light on sources of these associations by investigating potential moderating effects of education and labor force participation. Our results clearly replicate the previously found positive association between cognitive ability and Emotional Stability and Openness and the negative relationship between Conscientiousness and cognitive ability. The correlation between cognitive ability and Openness was found to be moderated by educational attainment, the negative association between Conscientiousness and cognitive ability was moderated by labor force participation

Beyond Competencies: Associations between Personality and School Grades Are Largely Independent of Subject-Specific and General Cognitive Competencies

The Big Five personality traits are established predictors of school grades. However, the mechanisms underlying these associations are not yet well understood. Effects of personality on grades might arise because behavioral tendencies facilitate learning and increase subject-specific competencies. Alternatively, personality effects on grades might be independent of cognitive competencies and reflect otherwise valued behaviors or teachers' grading practices. In the current study, we drew on large-scale data of 7th and 9th graders in Germany to explore the extent to which personality predicted grades even after accounting for competencies. Controlling for competencies and other key covariates, we cross-sectionally and longitudinally examined personality-grade associations across different school subjects, grade levels, and school types. Results indicate that the predictive power of personality is largely independent of subject-specific and general cognitive competencies. The largest effects emerged for conscientiousness. For openness, associations with grades partly overlapped with competencies, suggesting that openness may operate by fostering competencies. Overall, our results suggest that the associations between personality and grades unfold mostly independently of course mastery. This finding underlines the socioemotional value of personality in the classroom and encourages a more fine-grained view of the interplay between personality, competencies, classroom behavior, and grades