Data and code availability statements in systematic reviews of interventions were often missing or inaccurate: a content analysis

Objectives: To estimate the frequency of data and code availability statements in a random sample of systematic reviews with metaanalysis of aggregate data, summarize the content of the statements and investigate how often data and code files were shared. Methods: We searched for systematic reviews with meta-analysis of aggregate data on the effects of a health, social, behavioral, or educational intervention that were indexed in PubMed, Education Collection via ProQuest, Scopus via Elsevier, or Social Sciences Citation Index and Science Citation Index Expanded via Web of Science during a 4-week period (between November 2, and December 2, 2020). Records were randomly sorted and screened independently by two authors until our target sample of 300 systematic reviews was reached. Two authors independently recorded whether a data or code availability statement (or both) appeared in each review and coded the content of the statements using an inductive approach. Results: Of the 300 included systematic reviews with meta-analysis, 86 (29%) had a data availability statement, and seven (2%) had both a data and code availability statement. In 12/93 (13%) data availability statements, authors stated that data files were available for download from the journal website or a data repository, which we verified as being true. While 39/93 (42%) authors stated data were available upon request, 37/93 (40%) implied that sharing of data files was not necessary or applicable to them, most often because ‘‘all data appear in the article’’ or ‘‘no datasets were generated or analyzed’’. Discussion: Data and code availability statements appear infrequently in systematic review manuscripts. Authors who do provide a data availability statement often incorrectly imply that data sharing is not applicable to systematic reviews. Our results suggest the need for various interventions to increase data and code sharing by systematic reviewers.Matthew J. Page, Phi-Yen Nguyen, Daniel G. Hamilton, Neal R. Haddaway, Raju Kanukula, David Moher, Joanne E. McKenzi

Changing patterns in reporting and sharing of review data in systematic reviews with meta-analysis of the effects of interventions: Cross sectional meta-research study

Objectives: To examine changes in completeness of reporting and frequency of sharing data, analytical code, and other review materials in systematic reviews over time; and factors associated with these changes. Design: Cross sectional meta-research study. Population: Random sample of 300 systematic reviews with meta-analysis of aggregate data on the effects of a health, social, behavioural, or educational intervention. Reviews were indexed in PubMed, Science Citation Index, Social Sciences Citation Index, Scopus, and Education Collection in November 2020. Main outcome measures: The extent of complete reporting and the frequency of sharing review materials in the systematic reviews indexed in 2020 were compared with 110 systematic reviews indexed in February 2014. Associations between completeness of reporting and various factors (eg, self-reported use of reporting guidelines, journal policies on data sharing) were examined by calculating risk ratios and 95% confidence intervals. Results: Several items were reported suboptimally among 300 systematic reviews from 2020, such as a registration record for the review (n=113; 38%), a full search strategy for at least one database (n=214; 71%), methods used to assess risk of bias (n=185; 62%), methods used to prepare data for meta-analysis (n=101; 34%), and source of funding for the review (n=215; 72%). Only a few items not already reported at a high frequency in 2014 were reported more frequently in 2020. No evidence indicated that reviews using a reporting guideline were more completely reported than reviews not using a guideline. Reviews published in 2020 in journals that mandated either data sharing or inclusion of data availability statements were more likely to share their review materials (eg, data, code files) than reviews in journals without such mandates (16/87 (18%) v 4/213 (2%)). Conclusion: Incomplete reporting of several recommended items for systematic reviews persists, even in reviews that claim to have followed a reporting guideline. Journal policies on data sharing might encourage sharing of review materials.Phi-Yen Nguyen, Raju Kanukula, Joanne E McKenzie, Zainab Alqaidoom, Sue E Brennan, Neal R Haddaway, Daniel G Hamilton, Sathya Karunananthan, Steve McDonald, David Moher, Shinichi Nakagawa, David Nunan, Peter Tugwell, Vivian A Welch, Matthew J Pag