Integrative Imaging Informatics for Cancer Research: Workflow Automation for Neuro-oncology (I3CR-WANO)

Efforts to utilize growing volumes of clinical imaging data to generate tumor evaluations continue to require significant manual data wrangling owing to the data heterogeneity. Here, we propose an artificial intelligence-based solution for the aggregation and processing of multisequence neuro-oncology MRI data to extract quantitative tumor measurements. Our end-to-end framework i) classifies MRI sequences using an ensemble classifier, ii) preprocesses the data in a reproducible manner, iii) delineates tumor tissue subtypes using convolutional neural networks, and iv) extracts diverse radiomic features. Moreover, it is robust to missing sequences and adopts an expert-in-the-loop approach, where the segmentation results may be manually refined by radiologists. Following the implementation of the framework in Docker containers, it was applied to two retrospective glioma datasets collected from the Washington University School of Medicine (WUSM; n = 384) and the M.D. Anderson Cancer Center (MDA; n = 30) comprising preoperative MRI scans from patients with pathologically confirmed gliomas. The scan-type classifier yielded an accuracy of over 99%, correctly identifying sequences from 380/384 and 30/30 sessions from the WUSM and MDA datasets, respectively. Segmentation performance was quantified using the Dice Similarity Coefficient between the predicted and expert-refined tumor masks. Mean Dice scores were 0.882 ($\pm$0.244) and 0.977 ($\pm$0.04) for whole tumor segmentation for WUSM and MDA, respectively. This streamlined framework automatically curated, processed, and segmented raw MRI data of patients with varying grades of gliomas, enabling the curation of large-scale neuro-oncology datasets and demonstrating a high potential for integration as an assistive tool in clinical practice

Journal editors’ perspectives on the roles and tasks of peer reviewers in biomedical journals: a qualitative study

Objective Peer reviewers of biomedical journals are expected to perform a large number of roles and tasks, some of which are seemingly contradictory or demonstrate incongruities between the respective positions of peer reviewers and journal editors. Our aim was to explore the perspectives, expectations and understanding of the roles and tasks of peer reviewers of journal editors from general and specialty biomedical journals.Design Qualitative study.Setting Worldwide.Participants 56 journal editors from biomedical journals, most of whom were editors-in-chief (n=39), male (n=40) and worked part-time (n=50) at journals from 22 different publishers.Methods Semistructured interviews with journal editors were conducted. Recruitment was based on purposive maximum variation sampling. Data were analysed thematically following the methodology by Braun and Clarke.Results Journal editors' understanding of the roles and partly of tasks of peer reviewers are profoundly shaped by each journal's unique context and characteristics, including financial and human resources and journal reputation or prestige. There was a broad agreement among journal editors on expected technical tasks of peer reviewers related to scientific aspects, but there were different expectations in the level of depth. We also found that most journal editors support the perspective that authorship experience is key to high-quality reviews, while formal training in peer reviewing is not.Conclusion These journal editors' accounts reveal issues of a social nature within the peer-review process related to missed opportunities for journal editors to engage with peer reviewers to clarify the expected roles and tasks. Further research is needed on actual performance of peer reviewers looking into the content of peer-reviewer reports to inform meaningful training interventions, journal policies and guidelines

A scoping review on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals

International audienceBACKGROUND:Although peer reviewers play a key role in the manuscript review process, their roles and tasks are poorly defined. Clarity around this issue is important as it may influence the quality of peer reviewer reports. This scoping review explored the roles and tasks of peer reviewers of biomedical journals.METHODS:Comprehensive literature searches were conducted in Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science from inception up to May 2017. There were no date and language restrictions. We also searched for grey literature. Studies with statements mentioning roles, tasks and competencies pertaining to the role of peer reviewers in biomedical journals were eligible for inclusion. Two reviewers independently performed study screening and selection. Relevant statements were extracted, collated and classified into themes.RESULTS:After screening 2763 citations and 600 full-text papers, 209 articles and 13 grey literature sources were included. A total of 1426 statements related to roles were extracted, resulting in 76 unique statements. These were grouped into 13 emergent themes: proficient experts in their field (3 items), dutiful/altruistic towards scientific community (7 items), familiar with journal (2 items), unbiased and ethical professionals (18 items), self-critical professionals (4 items), reliable professionals (7 items), skilled critics (15 items), respectful communicators (6 items), gatekeepers (2 items), educators (2 items), advocates for author/editor/reader (3 items) and advisors to editors (2 items). Roles that do not fall within the remit of peer reviewers were also identified (5 items). We also extracted 2026 statements related to peer reviewers' tasks, resulting in 73 unique statements. These were grouped under six themes: organisation and approach to reviewing (10 items), make general comments (10 items), assess and address content for each section of the manuscript (36 items), address ethical aspects (5 items), assess manuscript presentation (8 items) and provide recommendations (4 items).CONCLUSIONS:Peer reviewers are expected to perform a large number of roles and tasks for biomedical journals. These warrant further discussion and clarification in order not to overburden these key actors

A scoping review protocol on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals

Introduction The primary functions of peer reviewers are poorly defined. Thus far no body of literature has systematically identified the roles and tasks of peer reviewers of biomedical journals. A clear establishment of these can lead to improvements in the peer review process. The purpose of this scoping review is to determine what is known on the roles and tasks of peer reviewers. Methods We will use the methodological framework first proposed by Arksey and O'Malley and subsequently adapted by Levac et al and the Joanna Briggs Institute. The scoping review will include all study designs, as well as editorials, commentaries and grey literature. The following eight electronic databases will be searched (from inception to May 2017): Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science. Two reviewers will use inclusion and exclusion criteria based on the 'Population-Concept-Context' framework to independently screen titles and abstracts of articles considered for inclusion. Full-text screening of relevant eligible articles will also be carried out by two reviewers. The search strategy for grey literature will include searching in websites of existing networks, biomedical journal publishers and organisations that offer resources for peer reviewers. In addition we will review journal guidelines to peer reviewers on how to perform the manuscript review. Journals will be selected using the 2016 journal impact factor. We will identify and assess the top five, middle five and lowest-ranking five journals across all medical specialties. Ethics and dissemination This scoping review will undertake a secondary analysis of data already collected and does not require ethical approval. The results will be disseminated through journals and conferences targeting stakeholders involved in peer review in biomedical research.Peer ReviewedPostprint (published version

Tools used to assess the quality of peer review reports: a methodological systematic review

Background A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research. Methods We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis. Results We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of ‘quality’. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer’s comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18). Conclusion Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research.Peer Reviewe

Development of ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research

Objective To develop a tool to assess the quality of peer- review reports in biomedical research. Methods We conducted an online survey intended for biomedical editors and authors. The survey aimed to (1) determine if participants endorse the proposed definition of peer- review report quality; (2) identify the most important items to include in the final version of the tool and (3) identify any missing items. Participants rated on a 5- point scale whether an item should be included in the tool and they were also invited to comment on the importance and wording of each item. Principal component analysis was performed to examine items redundancy and a general inductive approach was used for qualitative data analysis. Results A total of 446 biomedical editors and authors participated in the survey. Participants were mainly male (65.9%), middle- aged (mean=50.3, SD=13) and with PhD degrees (56.4%). The majority of participants (84%) agreed on the definition of peer- review report quality we proposed. The 20 initial items included in the survey questionnaire were generally highly rated with a mean score ranging from 3.38 (SD=1.13) to 4.60 (SD=0.69) (scale 1–5). Participants suggested 13 items that were not included in the initial list of items. A steering committee composed of five members with different expertise discussed the selection of items to include in the final version of the tool. The final checklist includes 14 items encompassed in five domains (Importance of the study, Robustness of the study methods, Interpretation and discussion of the study results, Reporting and transparency of the manuscript, Characteristics of peer reviewer’s comments). Conclusion Assessment of Review reports with a Checklist Available to eDItors and Authors tool could be used regularly by editors to evaluate the reviewers’ work, and also as an outcome when evaluating interventions to improve the peer- review process.Peer ReviewedPostprint (published version