The Process of Replication Target Selection in Psychology: What to Consider?

Increased execution of replication studies contributes to the effort to restore credibility of empirical research. However, a second generation of problems arises: the number of potential replication targets is at a serious mismatch with available resources. Given limited resources, replication target selection should be well justified, systematic, and transparently communicated. At present the discussion on what to consider when selecting a replication target is limited to theoretical discussion, self-reported justifications, and a few formalized suggestions. In this Registered Report, we proposed a study involving the scientific community to create a list of considerations for consultation when selecting a replication target in psychology. We employed a modified Delphi approach. First, we constructed a preliminary list of considerations. Second, we surveyed psychologists who previously selected a replication target with regards to their considerations. Third, we incorporated the results into the preliminary list of considerations and sent the updated list to a group of individuals knowledgeable about concerns regarding replication target selection. Over the course of several rounds, we established consensus regarding what to consider when selecting a replication target

Severe Osteogenesis Imperfecta in Cyclophilin B–Deficient Mice

Osteogenesis Imperfecta (OI) is a human syndrome characterized by exquisitely fragile bones due to osteoporosis. The majority of autosomal dominant OI cases result from point or splice site mutations in the type I collagen genes, which are thought to lead to aberrant osteoid within developing bones. OI also occurs in humans with homozygous mutations in Prolyl-3-Hydroxylase-1 (LEPRE1). Although P3H1 is known to hydroxylate a single residue (pro-986) in type I collagen chains, it is unclear how this modification acts to facilitate collagen fibril formation. P3H1 exists in a complex with CRTAP and the peptidyl-prolyl isomerase cyclophilin B (CypB), encoded by the Ppib gene. Mutations in CRTAP cause OI in mice and humans, through an unknown mechanism, while the role of CypB in this complex has been a complete mystery. To study the role of mammalian CypB, we generated mice lacking this protein. Early in life, Ppib-/- mice developed kyphosis and severe osteoporosis. Collagen fibrils in Ppib-/- mice had abnormal morphology, further consistent with an OI phenotype. In vitro studies revealed that in CypB–deficient fibroblasts, procollagen did not localize properly to the golgi. We found that levels of P3H1 were substantially reduced in Ppib-/- cells, while CRTAP was unaffected by loss of CypB. Conversely, knockdown of either P3H1 or CRTAP did not affect cellular levels of CypB, but prevented its interaction with collagen in vitro. Furthermore, knockdown of CRTAP also caused depletion of cellular P3H1. Consistent with these changes, post translational prolyl-3-hydroxylation of type I collagen by P3H1 was essentially absent in CypB–deficient cells and tissues from CypB–knockout mice. These data provide significant new mechanistic insight into the pathophysiology of OI and reveal how the members of the P3H1/CRTAP/CypB complex interact to direct proper formation of collagen and bone

Registered Replication Report on Fischer, Castel, Dodd, and Pratt (2003)

The attentional spatial-numerical association of response codes (Att-SNARC) effect (Fischer, Castel, Dodd, & Pratt, 2003)—the finding that participants are quicker to detect left-side targets when the targets are preceded by small numbers and quicker to detect right-side targets when they are preceded by large numbers—has been used as evidence for embodied number representations and to support strong claims about the link between number and space (e.g., a mental number line). We attempted to replicate Experiment 2 of Fischer et al. by collecting data from 1,105 participants at 17 labs. Across all 1,105 participants and four interstimulus-interval conditions, the proportion of times the effect we observed was positive (i.e., directionally consistent with the original effect) was 50. Further, the effects we observed both within and across labs were minuscule and incompatible with those observed by Fischer et al. Given this, we conclude that we failed to replicate the effect reported by Fischer et al. In addition, our analysis of several participant-level moderators (finger-counting habits, reading and writing direction, handedness, and mathematics fluency and mathematics anxiety) revealed no substantial moderating effects. Our results indicate that the Att-SNARC effect cannot be used as evidence to support strong claims about the link between number and space

A Novel Mutation in LEPRE1 That Eliminates Only the KDEL ER- Retrieval Sequence Causes Non-Lethal Osteogenesis Imperfecta

Prolyl 3-hydroxylase 1 (P3H1), encoded by the LEPRE1 gene, forms a molecular complex with cartilage-associated protein (CRTAP) and cyclophilin B (encoded by PPIB) in the endoplasmic reticulum (ER). This complex is responsible for one step in collagen post-translational modification, the prolyl 3-hydroxylation of specific proline residues, specifically α1(I) Pro986. P3H1 provides the enzymatic activity of the complex and has a Lys-Asp-Glu-Leu (KDEL) ER-retrieval sequence at the carboxyl terminus. Loss of function mutations in LEPRE1 lead to the Pro986 residue remaining unmodified and lead to slow folding and excessive helical post-translational modification of type I collagen, which is seen in both dominant and recessive osteogenesis imperfecta (OI). Here, we present the case of siblings with non-lethal OI due to novel compound heterozygous mutations in LEPRE1 (c.484delG and c.2155dupC). The results of RNA analysis and real-time PCR suggest that mRNA with c.2155dupC escapes from nonsense-mediated RNA decay. Without the KDEL ER- retrieval sequence, the product of the c.2155dupC variant cannot be retained in the ER. This is the first report of a mutation in LEPRE1 that eliminates only the KDEL ER-retrieval sequence, whereas other functional domains remain intact. Our study shows, for the first time, that the KDEL ER- retrieval sequence is essential for P3H1 functionality and that a defect in KDEL is sufficient for disease onset

Insights on the Evolution of Prolyl 3-Hydroxylation Sites from Comparative Analysis of Chicken and Xenopus Fibrillar Collagens

Recessive mutations that prevent 3-hydroxyproline formation in type I collagen have been shown to cause forms of osteogenesis imperfecta. In mammals, all A-clade collagen chains with a GPP sequence at the A1 site (P986), except α1(III), have 3Hyp at residue P986. Available avian, amphibian and reptilian type III collagen sequences from the genomic database (Ensembl) all differ in sequence motif from mammals at the A1 site. This suggests a potential evolutionary distinction in prolyl 3-hydroxylation between mammals and earlier vertebrates. Using peptide mass spectrometry, we confirmed that this 3Hyp site is fully occupied in α1(III) from an amphibian, Xenopus laevis, as it is in chicken. A thorough characterization of all predicted 3Hyp sites in collagen types I, II, III and V from chicken and xenopus revealed further differences in the pattern of occupancy of the A3 site (P707). In mammals only α2(I) and α2(V) chains had any 3Hyp at the A3 site, whereas in chicken all α-chains except α1(III) had A3 at least partially 3-hydroxylated. The A3 site was also partially 3-hydroxylated in xenopus α1(I). Minor differences in covalent cross-linking between chicken, xenopus and mammal type I and III collagens were also found as a potential index of evolving functional differences. The function of 3Hyp is still unknown but observed differences in site occupancy during vertebrate evolution are likely to give important clues

Deciding what to replicate: a formal definition of “replication value” and a decision model for replication study selection

Multivariate analysis of psychological dat

Registered Replication Report on Fischer, Castel, Dodd, and Pratt (2003)

The attentional spatial-numerical association of response codes (Att-SNARC) effect (Fischer, Castel, Dodd, & Pratt, 2003)—the finding that participants are quicker to detect left-side targets when the targets are preceded by small numbers and quicker to detect right-side targets when they are preceded by large numbers—has been used as evidence for embodied number representations and to support strong claims about the link between number and space (e.g., a mental number line). We attempted to replicate Experiment 2 of Fischer et al. by collecting data from 1,105 participants at 17 labs. Across all 1,105 participants and four interstimulus-interval conditions, the proportion of times the effect we observed was positive (i.e., directionally consistent with the original effect) was .50. Further, the effects we observed both within and across labs were minuscule and incompatible with those observed by Fischer et al. Given this, we conclude that we failed to replicate the effect reported by Fischer et al. In addition, our analysis of several participant-level moderators (finger-counting habits, reading and writing direction, handedness, and mathematics fluency and mathematics anxiety) revealed no substantial moderating effects. Our results indicate that the Att-SNARC effect cannot be used as evidence to support strong claims about the link between number and space

Registered replication report on Fischer, Castel, Dodd, and Pratt (2003)

The attentional spatial-numerical association of response codes (Att-SNARC) effect (Fischer, Castel, Dodd, & Pratt, 2003)—the finding that participants are quicker to detect left-side targets when the targets are preceded by small numbers and quicker to detect right-side targets when they are preceded by large numbers—has been used as evidence for embodied number representations and to support strong claims about the link between number and space (e.g., a mental number line). We attempted to replicate Experiment 2 of Fischer et al. by collecting data from 1,105 participants at 17 labs. Across all 1,105 participants and four interstimulus-interval conditions, the proportion of times the effect we observed was positive (i.e., directionally consistent with the original effect) was .50. Further, the effects we observed both within and across labs were minuscule and incompatible with those observed by Fischer et al. Given this, we conclude that we failed to replicate the effect reported by Fischer et al. In addition, our analysis of several participant-level moderators (finger-counting habits, reading and writing direction, handedness, and mathematics fluency and mathematics anxiety) revealed no substantial moderating effects. Our results indicate that the Att-SNARC effect cannot be used as evidence to support strong claims about the link between number and space

Investigating variation in replicability

Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation in the replicability of 13 classic and contemporary effects across 36 independent samples totaling 6,344 participants. In the aggregate, 10 effects replicated consistently. One effect – imagined contact reducing prejudice – showed weak support for replicability. And two effects – flag priming influencing conservatism and currency priming influencing system justification – did not replicate. We compared whether the conditions such as lab versus online or US versus international sample predicted effect magnitudes. By and large they did not. The results of this small sample of effects suggest that replicability is more dependent on the effect itself than on the sample and setting used to investigate the effect

The general fault in our fault lines

Pervading global narratives suggest that political polarization is increasing, yet the accuracy of such group meta-perceptions has been drawn into question. A recent US study suggests that these beliefs are inaccurate and drive polarized beliefs about out-groups. However, it also found that informing people of inaccuracies reduces those negative beliefs. In this work, we explore whether these results generalize to other countries. To achieve this, we replicate two of the original experiments with 10,207 participants across 26 countries. We focus on local group divisions, which we refer to as fault lines. We find broad generalizability for both inaccurate meta-perceptions and reduced negative motive attribution through a simple disclosure intervention. We conclude that inaccurate and negative group meta-perceptions are exhibited in myriad contexts and that informing individuals of their misperceptions can yield positive benefits for intergroup relations. Such generalizability highlights a robust phenomenon with implications for political discourse worldwide