Hawai'i Connecting to Collections Final Report

Hawai'iInstitute of Museum and Library Service

Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance (p < .05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion (p < .0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely high-powered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohen’s ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than .20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above .10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and low WEIRDness scores, respectively). Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied

Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance (p < .05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion (p < .0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely high-powered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohen’s ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than .20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above .10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and low WEIRDness scores, respectively). Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied

Colorectal cancer incidences in Lynch syndrome: a comparison of results from the prospective lynch syndrome database and the international mismatch repair consortium

Abstract Objective To compare colorectal cancer (CRC) incidences in carriers of pathogenic variants of the MMR genes in the PLSD and IMRC cohorts, of which only the former included mandatory colonoscopy surveillance for all participants. Methods CRC incidences were calculated in an intervention group comprising a cohort of confirmed carriers of pathogenic or likely pathogenic variants in mismatch repair genes (path_MMR) followed prospectively by the Prospective Lynch Syndrome Database (PLSD). All had colonoscopy surveillance, with polypectomy when polyps were identified. Comparison was made with a retrospective cohort reported by the International Mismatch Repair Consortium (IMRC). This comprised confirmed and inferred path_MMR carriers who were first- or second-degree relatives of Lynch syndrome probands. Results In the PLSD, 8,153 subjects had follow-up colonoscopy surveillance for a total of 67,604 years and 578 carriers had CRC diagnosed. Average cumulative incidences of CRC in path_MLH1 carriers at 70 years of age were 52% in males and 41% in females; for path_MSH2 50% and 39%; for path_MSH6 13% and 17% and for path_PMS2 11% and 8%. In contrast, in the IMRC cohort, corresponding cumulative incidences were 40% and 27%; 34% and 23%; 16% and 8% and 7% and 6%. Comparing just the European carriers in the two series gave similar findings. Numbers in the PLSD series did not allow comparisons of carriers from other continents separately. Cumulative incidences at 25 years were < 1% in all retrospective groups. Conclusions Prospectively observed CRC incidences (PLSD) in path_MLH1 and path_MSH2 carriers undergoing colonoscopy surveillance and polypectomy were higher than in the retrospective (IMRC) series, and were not reduced in path_MSH6 carriers. These findings were the opposite to those expected. CRC point incidence before 50 years of age was reduced in path_PMS2 carriers subjected to colonoscopy, but not significantly so