A many-analysts approach to the relation between religiosity and well-being

The relation between religiosity and well-being is one of the most researched topics in the psychology of religion, yet the directionality and robustness of the effect remains debated. Here, we adopted a many-analysts approach to assess the robustness of this relation based on a new cross-cultural dataset (N=10,535 participants from 24 countries). We recruited 120 analysis teams to investigate (1) whether religious people self-report higher well-being, and (2) whether the relation between religiosity and self-reported well-being depends on perceived cultural norms of religion (i.e., whether it is considered normal and desirable to be religious in a given country). In a two-stage procedure, the teams first created an analysis plan and then executed their planned analysis on the data. For the first research question, all but 3 teams reported positive effect sizes with credible/confidence intervals excluding zero (median reported β=0.120). For the second research question, this was the case for 65% of the teams (median reported β=0.039). While most teams applied (multilevel) linear regression models, there was considerable variability in the choice of items used to construct the independent variables, the dependent variable, and the included covariates

Unraveling the relation between representational competence and conceptual knowledge across four samples from two different countries

IntroductionWhereas it is commonly assumed that in learning science, representational competence is a critical prerequisite for the acquisition of conceptual knowledge, comprehensive psychometric investigations of this assumption are rare. We undertake a step in this direction by re-analyzing the data from a recent study that found a substantial correlation between the two constructs in undergraduates in the context of field representations and electromagnetism. MethodsWe re-analyze the data (N = 515 undergraduate students; M-age = 21.81, SDage = 4.04) to examine whether the relation between representational competence and conceptual knowledge, both measured with psychometrically validated test instruments, is similar or varies between four samples from two countries. To this end, we will employ correlational analysis and scatter plots. Employing these methods, we will examine whether a positive relation between representational competence and conceptual knowledge can be found and is of similar magnitude in all samples. We will also employ multiple-group latent profile analysis to examine how the more detailed association between the two constructs varies or is similar across samples. Finally, we will examine how commonalities and differences between samples relate to aspects of learners' gender, topic-specific learning opportunities, and individual preferences for specific learning content. The aim is to unravel the generalizability of this relation and thereby derive hypotheses for potential moderating factors that can be further examined in future research.ISSN:2504-284

Improving the utility of non-significant results for educational research: A review and recommendations

When used appropriately, non-significant p-values have the potential to further our understanding of what does not work in education, and why. When misinterpreted, they can trigger misguided conclusions, for example about the absence of an effect of an educational intervention, or about a difference in the efficacy of different interventions. We examined the frequency of non-significant p-values in recent volumes of peer-reviewed educational research journals. We also examined how frequently researchers misinterpret non-significance to imply the absence of an effect, or a difference to another significant effect. Within a random sample of 50 peer-reviewed articles, we found that of 528 statistically tested hypotheses, 253 (48%) were non-significant. Of these, 142 (56%) were erroneously interpreted to indicate the absence of an effect, and 59 (23%) to indicate a difference to another significant effect. For 97 (38%) of non-significant results, such misinterpretations were linked to potentially misguided implications for educational theory, practice, or policy. We outline valid ways for dealing with non-significant p-values to improve their utility for education, discussing potential reasons for these misinterpretations and implications for research.ISSN:1747-938

Longitudinal Within-Person Associations Between Quality of Social Relations, Structure of Social Relations, and Cognitive Functioning in Older Age

OBJECTIVES: Individuals' social connections and interpersonal experiences can both shape and be shaped by cognitive functioning. This study examines longitudinal within-person associations between quality of social relations, structure of social relations, and cognitive functioning in older age. METHOD: We examined 16-year longitudinal data (3 waves) from 497 older adults (M = 66.07 years, SD = 0.83, range = 64-68 years) from the Interdisciplinary Longitudinal Study of Adult Development and Aging (ILSE). Quality of social relations was measured by scales on perceived emotional support, instrumental support, and social integration. Structure of social relations was measured by self-reported number of leisure time partner types, indicating social network diversity. Cognitive functioning was assessed as a latent construct consisting of five cognitive tests (i.e., Information, Similarities, Letter Fluency, Picture Completion, Block Design). We used a random intercept cross-lagged panel model in the analysis. RESULTS: At the within-person level, prior quality of social relations, but not structure of social relations, was positively associated with subsequent cognitive functioning. Moreover, prior cognitive functioning was positively associated with subsequent structure of social relations, but not with quality of social relations. DISCUSSION: Quality of social relations is a protective factor of cognitive aging. Additionally, responding to prior lower cognitive functioning, social network diversity reduced, but quality of social relations did not seem to change. Overall, this study suggested that social relations and cognitive functioning mutually influence each other, but different aspects of social relations (i.e., quality, structure) might have different directional associations with cognitive functioning

Wachstumsverläufe der intrinsischen Wertüberzeugungen in Mathematik und Französisch: Zusammenhänge mit Berufsorientierungen

In dieser Längsschnittstudie wurden unterschiedliche Verläufe in der Entwicklung der intrinsischen Wertüberzeugungen in den Unterrichtsfächern Mathematik und Französisch in den Schuljahren 9 bis 11 und deren Zusammenhänge mit Berufsaspirationen untersucht. Bei 850 Schüler*innen von Deutschschweizer Gymnasien (54 % weiblich, Alter T1: 15,6 Jahre) wurden in einem bivariaten Wachstumsmodell fünf Wachstumsklassen identifiziert. Zwei dieser Klassen zeigten eine klare Differenzierung in den Wertüberzeugungen zwischen den beiden Fächern und eine stabile Entwicklung im bevorzugten Fach. Die anderen drei Klassen waren durch Mittelwertunterschiede (hohe, mittlere, niedrige intrinsische Wertüberzeugungen) gekennzeichnet und zeigten moderate Abnahmen über die Zeit in beiden Fächern. Die fünf Wachstumsklassen gingen mit unterschiedlichen Berufsorientierungen am Ende des 11. Schuljahres einher, wobei Berufsorientierungen bezüglich eines Fachs besonders hoch ausgeprägt waren, wenn die Wertüberzeugung bezüglich des anderen Faches niedrig war. Geschlechterunterschiede in den Berufsorientierungen konnten vollständig über die Zugehörigkeit der Geschlechter zu den fünf Wachstumsklassen erklärt werden

Same data, different conclusions : radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis

In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists’ gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for organizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed

Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis

In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists’ gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for organizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed