28 research outputs found
Investigating variation in replicability
Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation in the replicability of 13 classic and contemporary effects across 36 independent samples totaling 6,344 participants. In the aggregate, 10 effects replicated consistently. One effect â imagined contact reducing prejudice â showed weak support for replicability. And two effects â flag priming influencing conservatism and currency priming influencing system justification â did not replicate. We compared whether the conditions such as lab versus online or US versus international sample predicted effect magnitudes. By and large they did not. The results of this small sample of effects suggest that replicability is more dependent on the effect itself than on the sample and setting used to investigate the effect
Many Labs 2: Investigating Variation in Replicability Across Samples and Settings
We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance (p < .05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion (p < .0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely highpowered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohenâs ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than .20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above .10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and low WEIRDness scores, respectively). Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied.UCR::VicerrectorĂa de InvestigaciĂłn::Unidades de InvestigaciĂłn::Ciencias Sociales::Instituto de Investigaciones PsicolĂłgicas (IIP
Recommended from our members
Access to a responsiveness to intervention model: does beginning intervention in kindergarten matter?
In this study, we tested the outcomes of access to a response to intervention (RtI) model in kindergarten or in first grade on end-of-Grade-2 reading achievement and placement in special education. Across five schools, 214 students who began having access to Tier 2 intervention in kindergarten or first grade were compared in Grades 1 and 2 with 208 cohort peers who were average readers and 102 historical control condition second grade poor readers who did not receive Tier 2 intervention. Results demonstrated significant effects on reading achievement for access to RtI in kindergarten at the end of first grade (effects averaged 0.48), but not in second grade, except for students who were English language learners (ELLs), who showed an advantage through the end of second grade. Students with access to RtI overall had significantly higher outcomes at the end of Grade 2 than students in the historical control, with no differences resulting from ELL status. No significant difference in the proportion of students placed in special education was noted; however, a greater proportion of the students found eligible as with learning disabilities had poor reading scores if they were placed after participating in RtI
Recommended from our members
Building Better Bridges: Teaching Adolescents Who Are Poor Readers in Eighth Grade to Comprehend History Text
Helping struggling readers to learn history content in middle school can be difficult due to heavy reading demands. In this study, researchers taught poor readers with and without disabilities in eighth grade to generate main idea statements; create, compare, and contrast paragraphs; and identify cause and effect relations, along with relevant multisyllabic word study and vocabulary, as they read history text. The 34 participating students included 14 with disabilities and 20 without disabilities, who scored below the 5th percentile in reading, on average. The results were compared across special education and English learner status and with 81 typical readers from the same classes who studied the same units of history. Treated students made significant gains in use of these strategies, and poor readers with and without disabilities performed similar to their typical reader classmates in two of the three strategies following instruction. The instructional routines for each strategy are described