Causal inference in multilevel designs

The general theory of causal effects (Steyer et al., 2009) is used to develop a theory of causal inference for multilevel designs - i.e., for designs in which the effects of treatments are evaluated on units nested within clusters - that extends and consolidates previous approaches. Two multilevel causality spaces for different classes of multilevel designs are used to define true-effect variables, average causal effects, conditional causal effects and prima-facie effects. Unbiasedness, as the weakest condition under which average and conditional causal effects are identified, and its sufficient conditions are outlined. Next, stability assumptions for causal inference in multilevel designs are discussed in relation to the general theory of causal effects and a taxonomy of multilevel designs is introduced. Building upon this theoretical framework, the generalized analysis of covariance (ANCOVA), that extends the conventional multilevel ANCOVA by identifying the average causal effect in the presence of interactions, is developed for non-randomized multilevel designs with treatment assignment at unit- and at the cluster-level. Two simulation studies tested several statistical implementations of the generalized ANCOVAs. The results showed that contextual effects have to be taken into account in the specification of adjustment models, that predictors have to be modeled as stochastic to obtain correct standard errors of the average causal effects and that the unreliability of the empirical cluster means has to be accounted for in designs with treatment assignment at the cluster-level. The statistical methods studied in the simulations were applied to two empirical examples from educational research to demonstrate the implementations in practice. Finally, the scope of the general theory of causal effects, the advantages and disadvantages of the generalized ANCOVA and alternative adjustment methods are discussed and an overview of further research needs is given

Dimensional Comparison Theory : Paradoxical relations between self-beliefs and achievements in multiple domains

The internal/external frame of reference (I/E) model posits paradoxical relations between achievement and self-concept in mathematics and verbal domains, in which achievement in each domain has a positive effect on self-concept in the matching domain (e.g., mathematics achievement on mathematics self-concept) but a negative (contrastive) effect on self-concept in the non-matching domain (e.g., mathematics achievement on verbal self-concept). Extending the I/E model, Dimensional Comparison Theory (DCT) posits that self-evaluations are based on dimensional comparisons (e.g., how my accomplishments in one domain compare with my accomplishments in another domain) as well as the more traditional social and temporal comparisons, and on other sources of information about one's accomplishments. Extending the traditional tests of the I/E model, DCT predicts strong contrast effects only for contrasting domains that are at the opposite ends of the theoretical continuum of academic self-concept (far comparisons: e.g., the negative effect of math achievement on verbal self-concept), but much weaker negative contrast or even positive assimilation effects for complementary domains that are close to each other (near domains: e.g., positive effects of math achievement on physics self-concept; positive effects of native language on foreign language self-concept). Here we illustrate new predictions, theoretical insights, and methodology associated with DCT based on multiple academic domains (native language, foreign language, history, biology, physics and math), showing significant contrast effects for far comparisons and significantly less contrast or assimilation effects for near domains

What to do when scalar invariance fails: The extended alignment method for multi-group factor analysis comparison of latent means across many groups

Scalar invariance is an unachievable ideal that in practice can only be approximated; often using potentially questionable approaches such as partial invariance based on a stepwise selection of parameter estimates with large modification indices. Study 1 demonstrates an extension of the power and flexibility of the alignment approach for comparing latent factor means in large-scale studies (30 OECD countries, 8 factors, 44 items, N = 249,840), for which scalar invariance is typically not supported in the traditional confirmatory factor analysis approach to measurement invariance(CFA-MI). Importantly, we introduce an alignment-within-CFA (AwC) approach, transforming alignment from a largely exploratory tool into a confirmatory tool, and enabling analyses that previously have not been possible with alignment (testing the invariance of uniquenesses and factor variances/covariances; multiple-group MIMIC models; contrasts on latent means) and structural equation models more generally. Specifically, it also allowed a comparison of gender differences in a 30-country MIMIC AwC (i.e., a SEM with gender as a covariate) and a 60-group AwC CFA (i.e., 30 countries × 2 genders) analysis. Study 2, a simulation study following up issues raised in Study 1, showed that latent means were more accurately estimated with alignment than with the scalar CFA-MI, and particularly with partial invariance scalar models based on the heavily criticized stepwise selection strategy. In summary, alignment augmented by AwC provides applied researchers from diverse disciplines considerable flexibility to address substantively important issues when the traditional CFA-MI scalar model does not fit the data

Impact of Social and Dimensional Comparisons on Student‘s Mathematical and English Subject-Interest at the Beginning of Secondary School

Recent studies have analyzed social and dimensional comparisons simultaneously in order to consider their impact on students' academic self-concept (e.g., Chiu, 2012). Thereby, social comparisons refer to comparisons with the achievement level of students' classmates, whereas dimensional comparisons comprise comparisons between students' individual achievements across different domains. This paper analyzes whether both achievement comparisons influence students' subject-interest in mathematics and English (as a first foreign language). The analyses are based on N = 1390 German fifth and sixth grade students who participated in the BiKS-8-14 longitudinal study. Using multi-level analyses, results indicate that students' competences influence their mathematical and English subject-interests, demonstrating the typical pattern of social and dimensional comparisons. Further, analyses reveal mediation effects by subject-specific grades and self-concepts. These findings also apply for the development of students' subject-interest from grade 5 to grade 6. Results are discussed with respect to their implications concerning theories of achievement comparisons and interest development

Közművelődési jelenségek és jelentések. Az ÁMK

Many classroom climate studies suffer from 2 critical problems: They (a) treat climate as a student-level (L1) variable in single-level analyses instead of a classroom-level (L2) construct in multilevel analyses; and (b) rely on manifest-variable models rather than on latent-variable models that control measurement error at L1 and L2, and sampling error in the aggregation of L1 ratings to form L2 constructs. On the basis of an analysis of 2,541 students in Grades 5 or 6 from 89 classrooms, the authors demonstrate doubly latent multilevel structural equation models that overcome both of these problems. The results show that L2 classroom climate (a higher-order factor representing classroom mastery goal orientation, challenge, and teacher caring) had positive effects on self-efficacy and achievement. The authors conclude with a discussion of related issues (e.g., the meaning of L2 constructs vs. L1 residuals, the dimensionality of climate constructs at L2) and guidelines for future research

School or Work? The Choice May Change Your Personality

According to the social-investment principle, entering new environments is associated with new social roles that influence people's behaviors. In this study, we examined whether young adults' personality development is differentially related to their choice of either an academic or a vocational pathway (i.e., entering an academic-track school or beginning vocational training). The personality constructs of interest were Big Five personality traits and vocational-interest orientations. We used a longitudinal study design and propensity-score matching to create comparable groups before they entered one of the pathways and then tested the differences between these groups 6 years later. We expected the vocational pathway to reinforce more mature behavior and curtail investigative interest. Results indicated that choosing the vocational compared with the academic pathway was associated with higher conscientiousness and less interest in investigative, social, and enterprising activities

Gender Stereotypes in a Children's Television Program: Effects on Girls' and Boys' Stereotype Endorsement, Math Performance, Motivational Dispositions, and Attitudes

Television programs are a central part of children's everyday lives. These programs often transmit stereotypes about gender roles such as “math is for boys and not for girls.” So far, however, it is unclear whether stereotypes that are embedded in television programs affect girls' and boys' performance, motivational dispositions, or attitudes. On the basis of research on expectancy-value theory and stereotype threat, we conducted a randomized study with a total of 335 fifth-grade students to address this question. As the experimental material, we used a television program that had originally been produced for a national TV channel. The program was designed to show children that math could be interesting and fun. In the experimental condition, the program included a gender stereotyped segment in which two girls who were frustrated with math copied their math homework from a male classmate. In the control condition, participants watched an equally long, neutral summary of the first part of the video. We investigated effects on boys' and girls' stereotype endorsement, math performance, and different motivational constructs to gain insights into differential effects. On the basis of prior research, we expected negative effects of watching the stereotypes on girls' performance, motivational dispositions, and attitudes. Effects on the same outcomes for boys as well as children's stereotype endorsement were explored as open questions. We pre-registered our research predictions and analyses before conducting the experiment. Our results provide partial support for short-term effects of gender stereotypes embedded in television programs: Watching the stereotypes embedded in the video increased boys' and girls' stereotype endorsement. Boys reported a higher sense of belonging but lower utility value after watching the video with the stereotypes. Boys' other outcome variables were not affected, and there were also no effects on girl's performance, motivational dispositions, or attitudes. Results offer initial insights into how even short segments involving gender stereotypes in television shows can influence girls' and boys' stereotype endorsement and how such stereotypes may constitute one factor that contributes to gender differences in the STEM fields