Test-based and grade-based prediction of success in Economics Masters. Findings from the National Educational Panel Study

Fachspezifische Studierfähigkeitstests haben sich auch im deutschsprachigen Raum als valide Prädiktoren des Bachelorstudienerfolgs und somit als hilfreiche standardisierte Auswahlinstrumente erwiesen. Anhand der Daten des Nationalen Bildungspanels (NEPS) wurde untersucht, ob sich ein vergleichbares Muster auch für das konsekutive Masterstudium zeigt. Die Analysestichprobe umfasste 118 Studierende der Wirtschaftswissenschaften, die zum Ende ihres Bachelorstudiums einen fachspezifischen Kompetenztest bearbeiteten und ein Masterstudium aufnahmen. Der Test zeigte eine niedrige Validität (r = -.23) bezüglich der Masternoten. Es zeigte sich dabei keine inkrementelle Validität über die Bachelornote hinaus. Bei gleichem Testscore waren Frauen im Schnitt um 0.17 Notenpunkte besser als Männer. Die Ergebnisse sprechen gegen den Einsatz des wissenslastigen Fähigkeitstests zur Studierendenauswahl im Master. Gelingensfaktoren für eine erfolgreiche Studierendenauswahl werden im Kontext von Prädiktorenkombinationen diskutiert. (DIPF/Orig.)Subject-specific competence tests have been proven to be valid predictors of academic success in Bachelor studies; such tests are therefore useful standardized tools in student selection procedures. Using data from the National Educational Panel Study (NEPS), we investigated whether the pattern is similar for academic success in subsequent Master studies. The analysis sample consisted of 118 economics students who took a subject-specific competence test at the end of their Bachelor degree and began a subsequent Master. The test showed a weak validity (r = –.23) regarding the Master grade point average (GPA). It had no incremental validity over the Bachelor GPA. Given equal test scores, women outperformed men by 0.17 grade points on average. The results suggest that the knowledge-based test is not suited for Master student selection. Success factors for student selection procedures are discussed in the context of combining multiple predictors. (DIPF/Orig.

Reliability and validity of PIRLS and TIMSS: does the response format matter?

Academic achievements are often assessed in written exams and tests using selection-type (e.g., multiple-choice; MC) and supply-type (e.g., constructed-response; CR) item response formats. The present article examines how MC items and CR items differ with regard to reliability and criterion validity in two educational large-scale assessments with fourth-graders. The reading items of PIRLS 2006 were compiled into MC scales, CR scales, and mixed scales. Scale reliabilities were estimated according to item response theory (international PIRLS sample; n = 119,413). MC showed smaller standard errors than CR around the reading proficiency mean, whereas CR was more reliable for low and high proficiency levels. In the German sample (n = 7,581), there was no format-specific differential validity (criterion: German grades, r ˜ .5; ?r = 0.01). The mathematics items of TIMSS 2007 (n = 160,922) showed similar reliability patterns. MC validity was slightly larger than CR validity (criterion: mathematics grades; n = 5,111; r ˜ .5, ?r = –0.02). Effects of format-specific test-extensions were very small in both studies. It seems that in PIRLS and TIMSS, reliability and validity do not depend substantially on response formats. Consequently, other response format characteristics (like the cost of development, administration, and scoring) should be considered when choosing between MC and CR.This research was prepared with the support of the German funds “Bund-Länder-Programm für bessere Studienbedingungen und mehr Qualität in der Lehre (‘Qualitätspakt Lehre’)” [the joint program of the Federal and States Government for better study conditions and the quality of teaching in higher education (“the Teaching Quality Pact”)] at Saarland University (funding code: 01PL11012). The authors developed the topic and the content of this manuscript independently from this funding. We thank the Institute for School Development Research (IFS) at Technical University Dortmund / the Max Planck Institute for Human Development (MPIB) Berlin / the Standing Conference of the Ministers of Education and Cultural Affairs (KMK) as well as the Research Data Centre (FDZ) at the Institute for Educational Quality Improvement (IQB) for providing the raw data

Aktuelle Entwicklungen in der deutschen Familiensoziologie: Theorien, Daten, Methoden

"Wie und in welchen Bereichen hat sich in den letzten 10 Jahren unser Wissen von familialen Strukturen und Dynamiken sowie den Ursachen ihres Wandels verbessert? Wo liegen die inhaltlichen Schwerpunkte der Forschung und welche Fragestellungen werden vernachlässigt? Welche Methoden werden in der empirischen Familienforschung verwendet, und welche Daten stehen zur Verfügung? Der Beitrag resümiert den Diskussions- und Forschungsstand zu diesen Fragen in der deutschen Familiensoziologie unter Bezugnahme auf die internationale Literatur. Neben einer Bestandsaufnahme der Forschungsschwerpunkte in den letzten 10 Jahren und einem Überblick über verfügbare Daten und Methoden fokussieren die Autoren auf die Identifikation von aktuellen inhaltlichen Forschungslücken und methodischen Defiziten. Es werden in drei Thesen Forderungen an die aktuelle sozialwissenschaftliche Familienforschung formuliert: Eine zeitgemäße, sozialpolitisch relevante familiensoziologische Forschung muss 1) neben strukturellen Faktoren stärker subjektive, kulturelle und soziale Einflussfaktoren familialen Wandels berücksichtigen; 2) mehr belastbares Wissen über die Leistungen und die Leistungsfähigkeit der Familie in unserer Gegenwartsgesellschaft gewinnen; und 3) sich stärker in angrenzende Forschungsbereiche einmischen, u.a. Bildung, Ungleichheit und Migration." (Autorenreferat)"How and in which areas did our knowledge of family structures, family dynamics and the determinants of family change improve in the past decade? Which substantive areas receive most attention and which questions are underresearched? Which methods are commonly applied in empirical family research and what can we say about data availability? This article reviews the current discussion and recent research on these questions in German family sociology in the context of the international literature. Next to a review of the substantive research foci in the past decade and an overview of available data and methods, we focus on identifying current substantive research gaps and methodological deficits. We formulate three requests for current family research in the social sciences: a timely family sociology that is relevant for social policy has to 1) pay more attention to subjective, cultural and social influences on family change beyond its structural determinants; 2) generate more knowledge on the contributions and capabilities of families in our contemporary society; and 3) intervene more forcefully into adjacent research areas including education, stratification and migration." (author's abstract

Competition as rational action : why young children cannot appreciate competitive games

Understanding rational actions requires perspective taking both with respect to means and with respect to objectives. This study addresses the question of whether the two kinds of perspective taking develop simultaneously or in sequence. It is argued that evidence from competitive behavior is best suited for settling this issue. A total of 71 kindergarten children between 3 and 5 years of age participated in a competitive game of dice and were tested on two traditional false belief stories as well as on several control tasks (verbal intelligence, inhibitory control, and working memory). The frequency of competitive poaching moves in the game correlated with correct predictions of mistaken actions in the false belief task. Hierarchical linear regression after controlling for age and control variables showed that false belief understanding significantly predicted the amount of poaching moves. The results speak for an interrelated development of the capacity for “instrumental” and “telic” perspective taking. They are discussed in the light of teleology as opposed to theory use and simulation

Categorial Differences in Affective Picture Perception

Arbeiten zur affektiven Modulation von Verhaltens- und physiologischen Parametern zeigen häufig einen Verarbeitungsvorteil von erregenden angenehmen und unangenehmen gegenüber neutralen Reizen. Davon ausgehend wurde die Erkennungsleistung von Bildern untersucht, wobei versucht wurde mögliche perzeptuelle Unterschiede zwischen den Kategorien zu minimieren. Eine Auswahl von jeweils 180 angenehmen, neutralen und unangenehmen Schwarzweißstimuli wurde verwendet. Die durchschnittliche Helligkeit und Komplexität der Bilder in diesen Valenzkategorien wurde kontrolliert. Ein sandwichmaskierter Zielreiz wurde präsentiert (13, 27 oder 40 ms). Anschließend mussten die Probanden entscheiden, ob es sich bei einem Kontrollbild um das Zielbild handelt oder nicht, sowie die subjektive Sicherheit ihrer Einschätzung angeben. Es zeigte sich ein linearer Effekt der Präsentationsdauer auf die Erkennungsleistung für alle Bildkategorien: Je länger der Zielreiz gezeigt wurde, desto mehr richtige Antworten gab es und desto kürzer waren die Antwortzeiten. Für die einzelnen Präsentationsdauern zeigte sich kein klarer Effekt der Zielbildvalenz auf die Erkennungsleistung. Bei drei der 19 Versuchspersonen zeigten sich in mindestens einer Präsentationsdauerbedingung signifikante Unterschiede zwischen den Valenzkategorien, die allerdings keine einheitliche Richtung hatten. In Durchgängen mit sehr erregenden Bildern gab es weniger richtige Antworten als in Durchgängen mit niedrig erregenden Bildern. Dieses Ergebnis deutet darauf hin, dass emotionale Prozesse, die in der Gegenwart von vielen Reizen, die um Aufmerksamkeit und Verarbeitung konkurrieren, vermutlich optimal ablaufen, die Bildidentifikation im aktuellen Experiment behindern

Once Upon a Time in the Test : Sex Differences in the Prediction of Academic Achievement and Job Performance

The present thesis answers several open questions regarding the gender fairness of scholastic aptitude tests and provides practical advice how to assess test fairness and minimize predictive bias.There are several reasons to use aptitude tests in the college admission process: they offer standardized scores, provide incremental validity over high school records, and may influence the educational decisions of applicants. Despite the usefulness of these tests in practice, their construct validity, the reasons for group differences and other psychometric aspects usually remain unclear. Consequently, a closer look at the fairness of admission tools reveals many gray areas where improper test use and imprecise conceptualization cannot be easily distinguished.Test fairness in a narrow, psychometric sense is based on the lack of systematic bias. Three types of bias are generally distinguished: differential item functioning (DIF; an item is more difficult (or easier) for a particular subgroup after controlling for the ability it is supposed to measure), differential validity (different criterion validities for subgroups), and differential prediction (performances of subgroups are systemically underpredicted (or overpredicted)).Four studies have been conducted to shed light on the extent and possible explanations of sex-specific bias associated with scholastic aptitude testing and the prediction of academic and vocational performance. In the first two studies, special attention was given to the role of intelligence facets, because general mental ability (g) and scholastic aptitude overlap conceptually - reasoning is among the constructs assessed by most college admission tests - and are highly correlated.Study 1 provides a detailed look at the situation in Germany. Three student samples show various levels of differential prediction. Across all samples, mathematical reasoning yields the most favorable predictions for men (i.e., men's college grades are overpredicted). High School Grade Point Average (HSGPA; "Abiturnote"), on the other hand, is the least favorable predictor for men's academic performance, although it still underpredicts women's performance in two of the samples.Study 2 explores the construct validity of two German tests of subject-specific scholastic aptitude. The link between intelligence and aptitude test score is confirmed. Small sex differences in validities suggest a stronger relationship between verbal reasoning and scholastic aptitude for women than for men.Study 3 broadens the scope by looking at the careers of university students two years after their graduation. Valid predictors for success at work include personal interests, occupational status, math grades, and conscientiousness. The gender pay gap remains even after controlling for socio-economic status and motivational factors.Study 4 demonstrates the aggregation of differential prediction findings with meta-analytical methods. The underprediction of women's college grades by aptitude tests can be reduced (but not eliminated) by using HSGPA and test scores as predictors. Graduate tests do not show differential prediction.Based on these findings, two promising explanations for differential prediction are scrutinized. On the one hand, sex differences in vocational interests exist which are associated with choice of major and career paths. On the other hand, women appear to approach academic challenges in a more holistic way than men, which interferes with their admission test performance, but facilitates their academic performance, eventually.Although some topics still need further attention (e.g., construct validity of grades, availability of large-scale data sets, socio-economic consequences of admission testing), my findings clarify the psychometric properties of scholastic aptitude tests and provide immediate suggestions for weighting subscales in order to maximize gender fairness

The Influence of Social Capital and Tolerance on Democratic Performance

The present paper deals with the questions to what extend does social capital influence democratic values directly and whether there is an indirect influence through interpersonal and political tolerance. Data from the German and the Swiss part of the European Values Survey (EVS) mid-90s wave are used for the empirical analysis. Conceptual and methodological effects influence the structural equation model. Individual social capital appears to have a moderate influence on democratic attitude, whereas the role of tolerance is of minor importance in Germany and neglectable in Switzerland. The results support the notion that social trust and civic associations foster democratic values. Both personal and political tolerance can be regarded as separate concepts with rather weak relationships in the given framework. The lack of influence of political tolerance on democratic attitude is remarkable, but can partly be attributed to the respective tolerance items, which are binary and do not differentiate between degrees of tolerance