Quality assessment of scientific manuscripts in peer review and education

We report a vignette study and a survey to investigate which study characteristics influence quality ratings academics give of articles submitted for publication, and academics and students give of students’ theses. In the vignette study, 800 respondents evaluated the quality of an abstract of studies with small or large sample sizes, showing statistically significant or non-significant results, and containing statistical reporting errors or no errors. In the survey, the same participants rated the importance of 29 manuscript characteristics related to the study’s theory, design, conduct, data analyses, and presentation for assessing either the quality of a manuscript or its publishability (article) or grade (thesis). Results showed that quality ratings were affected by sample sizes but not by statistical significance or the presence of statistical reporting errors in the rated researchvignette. These results suggest that researchers’ assessments of manuscript quality are not responsible for publication bias. Furthermore, academics and students provided highly similar ratings of the importance of different aspects relevant to quality assessment of articles and theses. These results suggest that quality criteria for scientific manuscripts are already adopted by students and are similar for submitted manuscripts and theses

Phenotyping of left and right ventricular function in mouse models of compensated hypertrophy and heart failure with cardiac MRI

Background: Left ventricular (LV) and right ventricular (RV) function have an important impact on symptom occurrence, disease progression and exercise tolerance in pressure overload-induced heart failure, but particularly RV functional changes are not well described in the relevant aortic banding mouse model. Therefore, we quantified time-dependent alterations in the ventricular morphology and function in two models of hypertrophy and heart failure and we studied the relationship between RV and LV function during the transition from hypertrophy to heart failure. Methods: MRI was used to quantify RV and LV function and morphology in healthy (n = 4) and sham operated (n = 3) C57BL/6 mice, and animals with a mild (n = 5) and a severe aortic constriction (n = 10). Results: Mice subjected to a mild constriction showed increased LV mass (P,0.01) and depressed LV ejection fraction (EF) (P,0.05) as compared to controls, but had similar RVEF (P.0.05). Animals with a severe constriction progressively developed LV hypertrophy (P,0.001), depressed LVEF (P,0.001), followed by a declining RVEF (P,0.001) and the development of pulmonary remodeling, as compared to controls during a 10-week follow-up. Myocardial strain, as a measure for local cardiac function, decreased in mice with a severe constriction compared to controls (P,0.05). Conclusions: Relevant changes in mouse RV and LV function following an aortic constriction could be quantified using MRI. The well-controlled models described here open opportunities to assess the added value of new MRI techniques for the diagnosis of heart failure and to study the impact of new therapeutic strategies on disease progression and symptom occurrence

Phenotyping of Left and Right Ventricular Function in Mouse Models of Compensated Hypertrophy and Heart Failure with Cardiac MRI

Background: Left ventricular (LV) and right ventricular (RV) function have an important impact on symptom occurrence, disease progression and exercise tolerance in pressure overload-induced heart failure, but particularly RV functional changes are not well described in the relevant aortic banding mouse model. Therefore, we quantified time-dependent alterations in the ventricular morphology and function in two models of hypertrophy and heart failure and we studied the relationship between RV and LV function during the transition from hypertrophy to heart failure. Methods: MRI was used to quantify RV and LV function and morphology in healthy (n = 4) and sham operated (n = 3) C57BL/6 mice, and animals with a mild (n = 5) and a severe aortic constriction (n = 10). Results: Mice subjected to a mild constriction showed increased LV mass (P0.05). Animals with a severe constriction progressively developed LV hypertrophy (P<0.001), depressed LVEF (P<0.001), followed by a declining RVEF (P<0.001) and the development of pulmonary remodeling, as compared to controls during a 10-week follow-up. Myocardial strain, as a measure for local cardiac function, decreased in mice with a severe constriction compared to controls (P<0.05). Conclusions: Relevant changes in mouse RV and LV function following an aortic constriction could be quantified using MRI. The well-controlled models described here open opportunities to assess the added value of new MRI techniques for the diagnosis of heart failure and to study the impact of new therapeutic strategies on disease progression and symptom occurrence

Longitudinal analysis of varicella-zoster virus-specific antibodies in systemic lupus erythematosus:No association with subclinical viral reactivations or lupus disease activity

Systemic lupus erythematosus (SLE) patients are at high risk of herpes zoster. Previously, we found increased immunoglobulin (Ig)G levels against varicella-zoster virus (VZV) in SLE patients compared to controls, while antibody levels against diphtheria and cellular immunity to VZV were decreased. We aimed to test our hypothesis that increased VZV-IgG levels in SLE result from subclinical VZV reactivations, caused by stress because of lupus disease activity or immunosuppressive drug use. Methods Antibody levels to VZV (IgG, IgA, IgM), total IgG and VZV-DNA were longitudinally determined in the serum of 34 SLE patients, using enzyme-linked immunosorbent assay and polymerase chain reaction. Clinical data were retrieved from medical records. Reactivation of VZV was defined as an at least fivefold rise in VZV-IgG or presence of VZV-IgM or VZV-DNA. Generalized estimating equations (GEE) were used to longitudinally analyse associations between antibody levels, lupus disease activity and medication use. Systemic Lupus Erythematosus Disease Activity Index, anti-double-stranded DNA and complement levels were used as indicators of lupus disease activity. Results A VZV reactivation was determined in 11 patients (33%). In at least five of them, herpes zoster was clinically overt. No association between SLE disease activity or medication use and VZV-specific antibody levels was found. There was a weak association between total IgG and VZV-IgG. Conclusions Our results indicate that increased VZV-IgG levels in SLE do not result from frequent subclinical VZV reactivations, and are not associated with lupus disease activity. Increased VZV-IgG can only partially be explained by hypergammaglobulinaemia

Ensuring the quality and specificity of preregistrations

Researchers face many, often seemingly arbitrary, choices in formulating hypotheses, designing protocols, collecting data, analyzing data, and reporting results. Opportunistic use of “researcher degrees of freedom” aimed at obtaining statistical significance increases the likelihood of obtaining and publishing false-positive results and overestimated effect sizes. Preregistration is a mechanism for reducing such degrees of freedom by specifying designs and analysis plans before observing the research outcomes. The effectiveness of preregistration may depend, in part, on whether the process facilitates sufficiently specific articulation of such plans. In this preregistered study, we compared 2 formats of preregistration available on the OSF: Standard Pre-Data Collection Registration and Prereg Challenge Registration (now called “OSF Preregistration,” http://osf.io/prereg/). The Prereg Challenge format was a “structured” workflow with detailed instructions and an independent review to confirm completeness; the “Standard” format was “unstructured” with minimal direct guidance to give researchers flexibility for what to prespecify. Results of comparing random samples of 53 preregistrations from each format indicate that the “structured” format restricted the opportunistic use of researcher degrees of freedom better (Cliff’s Delta = 0.49) than the “unstructured” format, but neither eliminated all researcher degrees of freedom. We also observed very low concordance among coders about the number of hypotheses (14%), indicating that they are often not clearly stated. We conclude that effective preregistration is challenging, and registration formats that provide effective guidance may improve the quality of research

Response to Comment on “Estimating the reproducibility of psychological science”

Gilbert et al. conclude that evidence from the Open Science Collaboration's Reproducibility Project: Psychology indicates high reproducibility, given the study methodology. Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data. Using the Reproducibility Project: Psychology data, both optimistic and pessimistic conclusions about reproducibility are possible, and neither are yet warranted.status: publishe

Consensus-based guidance for conducting and reporting multi-analyst studies

International audienceAny large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research