Seven easy steps to open science: An annotated reading list

The open science movement is rapidly changing the scientific landscape. Because exact definitions are often lacking and reforms are constantly evolving, accessible guides to open science are needed. This paper provides an introduction to open science and related reforms in the form of an annotated reading list of seven peer-reviewed articles, following the format of Etz, Gronau, Dablander, Edelsbrunner, and Baribault (2018). Written for researchers and students-particularly in psychological science-it highlights and introduces seven topics: understanding open science; open access; open data, materials, and code; reproducible analyses; preregistration and registered reports; replication research; and teaching open science. For each topic, we provide a detailed summary of one particularly informative and actionable article and suggest several further resources. Supporting a broader understanding of open science issues, this overview should enable researchers to engage with, improve, and implement current open, transparent, reproducible, replicable, and cumulative scientific practices

What You Find Depends on How You Measure It: Reactivity of Response Scales Measuring Predecisional Information Distortion in Medical Diagnosis

“Predecisional information distortion” occurs when decision makers evaluate new information in a way that is biased towards their leading option. The phenomenon is well established, as is the method typically used to measure it, termed “stepwise evolution of preference” (SEP). An inadequacy of this method has recently come to the fore: it measures distortion as the total advantage afforded a leading option over its competitor, and therefore it cannot differentiate between distortion to strengthen a leading option (“proleader” distortion) and distortion to weaken a trailing option (“antitrailer” distortion). To address this, recent research introduced new response scales to SEP. We explore whether and how these new response scales might influence the very proleader and antitrailer processes that they were designed to capture (“reactivity”). We used the SEP method with concurrent verbal reporting: fifty family physicians verbalized their thoughts as they evaluated patient symptoms and signs (“cues”) in relation to two competing diagnostic hypotheses. Twenty-five physicians evaluated each cue using the response scale traditional to SEP (a single response scale, returning a single measure of distortion); the other twenty-five did so using the response scales introduced in recent studies (two separate response scales, returning two separate measures of distortion: proleader and antitrailer). We measured proleader and antitrailer processes in verbalizations, and compared verbalizations in the single-scale and separate-scales groups. Response scales did not appear to affect proleader processes: the two groups of physicians were equally likely to bolster their leading diagnosis verbally. Response scales did, however, appear to affect antitrailer processes: the two groups denigrated their trailing diagnosis verbally to differing degrees. Our findings suggest that the response scales used to measure information distortion might influence its constituent processes, limiting their generalizability across and beyond experimental studies

Registered Replication Report: Dijksterhuis and van Knippenberg (1998)

Dijksterhuis and van Knippenberg (1998) reported that participants primed with a category associated with intelligence ("professor") subsequently performed 13% better on a trivia test than participants primed with a category associated with a lack of intelligence ("soccer hooligans"). In two unpublished replications of this study designed to verify the appropriate testing procedures, Dijksterhuis, van Knippenberg, and Holland observed a smaller difference between conditions (2%-3%) as well as a gender difference: Men showed the effect (9.3% and 7.6%), but women did not (0.3% and -0.3%). The procedure used in those replications served as the basis for this multilab Registered Replication Report. A total of 40 laboratories collected data for this project, and 23 of these laboratories met all inclusion criteria. Here we report the meta-analytic results for those 23 direct replications (total N = 4,493), which tested whether performance on a 30-item general-knowledge trivia task differed between these two priming conditions (results of supplementary analyses of the data from all 40 labs, N = 6,454, are also reported). We observed no overall difference in trivia performance between participants primed with the "professor" category and those primed with the "hooligan" category (0.14%) and no moderation by gender

Eye tracking: empirical foundations for a minimal reporting guideline

In this paper, we present a review of how the various aspects of any study using an eye tracker (such as the instrument, methodology, environment, participant, etc.) affect the quality of the recorded eye-tracking data and the obtained eye-movement and gaze measures. We take this review to represent the empirical foundation for reporting guidelines of any study involving an eye tracker. We compare this empirical foundation to five existing reporting guidelines and to a database of 207 published eye-tracking studies. We find that reporting guidelines vary substantially and do not match with actual reporting practices. We end by deriving a minimal, flexible reporting guideline based on empirical research (Section "empirically based minimal reporting guideline")

Body appreciation around the world: Measurement invariance of the Body Appreciation Scale-2 (BAS-2) across 65 nations, 40 languages, gender identities, and age.

The Body Appreciation Scale-2 (BAS-2) is a widely used measure of a core facet of the positive body image construct. However, extant research concerning measurement invariance of the BAS-2 across a large number of nations remains limited. Here, we utilised the Body Image in Nature (BINS) dataset - with data collected between 2020 and 2022 - to assess measurement invariance of the BAS-2 across 65 nations, 40 languages, gender identities, and age groups. Multi-group confirmatory factor analysis indicated that full scalar invariance was upheld across all nations, languages, gender identities, and age groups, suggesting that the unidimensional BAS-2 model has widespread applicability. There were large differences across nations and languages in latent body appreciation, while differences across gender identities and age groups were negligible-to-small. Additionally, greater body appreciation was significantly associated with higher life satisfaction, being single (versus being married or in a committed relationship), and greater rurality (versus urbanicity). Across a subset of nations where nation-level data were available, greater body appreciation was also significantly associated with greater cultural distance from the United States and greater relative income inequality. These findings suggest that the BAS-2 likely captures a near-universal conceptualisation of the body appreciation construct, which should facilitate further cross-cultural research. [Abstract copyright: Copyright © 2023 The Authors. Published by Elsevier Ltd.. All rights reserved.

Sunlight exposure exerts immunomodulatory effects to reduce multiple sclerosis severity

Multiple sclerosis (MS) disease risk is associated with reduced sun-exposure. This study assessed the relationship between measures of sun exposure (vitamin D [vitD], latitude) and MS severity in the setting of two multicenter cohort studies (n(NationMS) = 946, n(BIONAT) = 990). Additionally, effect-modification by medication and photosensitivity-associated MC1R variants was assessed. High serum vitD was associated with a reduced MS severity score (MSSS), reduced risk for relapses, and lower disability accumulation over time. Low latitude was associated with higher vitD, lower MSSS, fewer gadolinium-enhancing lesions, and lower disability accumulation. The association of latitude with disability was lacking in IFN-β-treated patients. In carriers of MC1R:rs1805008(T), who reported increased sensitivity toward sunlight, lower latitude was associated with higher MRI activity, whereas for noncarriers there was less MRI activity at lower latitudes. In a further exploratory approach, the effect of ultraviolet (UV)-phototherapy on the transcriptome of immune cells of MS patients was assessed using samples from an earlier study. Phototherapy induced a vitD and type I IFN signature that was most apparent in monocytes but that could also be detected in B and T cells. In summary, our study suggests beneficial effects of sun exposure on established MS, as demonstrated by a correlative network between the three factors: Latitude, vitD, and disease severity. However, sun exposure might be detrimental for photosensitive patients. Furthermore, a direct induction of type I IFNs through sun exposure could be another mechanism of UV-mediated immune-modulation in MS

Body appreciation around the world: Measurement invariance of the Body Appreciation Scale-2 (BAS-2) across 65 nations, 40 languages, gender identities, and age

Abstract The Body Appreciation Scale-2 (BAS-2) is a widely used measure of a core facet of the positive body image construct. However, extant research concerning measurement invariance of the BAS-2 across a large number of nations remains limited. Here, we utilised the Body Image in Nature (BINS) dataset – with data collected between 2020 and 2022 – to assess measurement invariance of the BAS-2 across 65 nations, 40 languages, gender identities, and age groups. Multi-group confirmatory factor analysis indicated that full scalar invariance was upheld across all nations, languages, gender identities, and age groups, suggesting that the unidimensional BAS-2 model has widespread applicability. There were large differences across nations and languages in latent body appreciation, while differences across gender identities and age groups were negligible-to-small. Additionally, greater body appreciation was significantly associated with higher life satisfaction, being single (versus being married or in a committed relationship), and greater rurality (versus urbanicity). Across a subset of nations where nation-level data were available, greater body appreciation was also significantly associated with greater cultural distance from the United States and greater relative income inequality. These findings suggest that the BAS-2 likely captures a near-universal conceptualisation of the body appreciation construct, which should facilitate further cross-cultural research

Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis

In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists’ gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for organizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed