Optimizing Radiology Peer Review: A Mathematical Model for Selecting Future Cases Based on Prior Errors

Introduction: Peer review is an essential process for physicians because it facilitates improved quality of patient care and continuing physician learning and improvement. However, peer review often is not well received by radiologists, who note that it is time intensive, subjective, and lacks demonstrable impact on patient care. Current advances in peer review include the RADPEER system with its standardization of discrepancies and incorporation of the peer review process into the PACS itself. Our purpose was to build on RADPEER and similar systems by using a mathematical model to optimally select the types of cases to be reviewed, for each radiologist undergoing review, based on the past frequency of interpretive error, likelihood of morbidity from an error, financial cost of an error, and time required for the reviewing radiologist to interpret the study. Methods: We compiled 612,890 preliminary radiology reports authored by residents and attendings of a large tertiary-care medical center from 1999 to 2004. Discrepancies between preliminary and final interpretations were classified by severity and validated by re-review of major discrepancies. A mathematical model was then used to calculate, for each author of a preliminary report, the combined morbidity and financial costs of expected errors across three modalities (MRI, CT, and CR) and four departmental divisions (Neuroradiology and Abdominal, Musculoskelatal, and Thoracic Imaging). Results: A customized report was generated for each on-call radiologist which determined the category (modality and body part) with the highest total cost function. A universal total cost based on probability data from all radiologists was also compiled. Conclusion: The use of mathematical models to guide case selection could optimize the efficiency and effectiveness of physician time spent on peer review and produce more concrete and meaningful feedback to radiologists undergoing peer review

Interrater Reliability of NI-RADS on Posttreatment PET/Contrast-enhanced CT Scans in Head and Neck Squamous Cell Carcinoma

Purpose: To evaluate the interrater reliability among radiologists examining posttreatment head and neck squamous cell carcinoma (HNSCC) fluorodeoxyglucose PET/contrast-enhanced CT (CECT) scans using Neck Imaging Reporting and Data System (NI-RADS). Materials and Methods: In this retrospective study, images in 80 patients with HNSCC who underwent posttreatment surveillance PET/CECT and immediate prior comparison CECT or PET/CECT (from June 2014 to July 2016) were uploaded to the American College of Radiology's cloud-based website, Cortex. Eight radiologists from seven institutions with variable NI-RADS experience independently evaluated each case and assigned an appropriate prose description and NI-RADS category for the primary site and the neck site. Five of these individuals were experienced readers (> 5 years of experience), and three were novices (< 5 years of experience). In total, 640 lexicon-based and NI-RADS categories were assigned to lesions among the 80 included patients by the eight radiologists. Light generalization of Cohen κ for interrater reliability was performed. Results: Of the 80 included patients (mean age, 63 years ± 10 [standard deviation]), there were 58 men (73%); 60 patients had stage IV HNSCC (75%), and the most common tumor location was oropharynx (n = 32; 40%). Light κ for lexicon was 0.30 (95% CI: 0.23, 0.36) at the primary site and 0.31 (95% CI: 0.24, 0.37) at the neck site. Light κ for NI-RADS category was 0.55 (95% CI: 0.46, 0.63) at the primary site and 0.60 (95% CI: 0.48, 0.69) at the neck site. Percent agreement between lexicon and correlative NI-RADS category was 84.4% (540 of 640) at the primary site and 92.6% (593 of 640) at the neck site. There was no significant difference in interobserver agreement among the experienced versus novice raters. Conclusion: Moderate agreement was achieved among eight radiologists using NI-RADS at posttreatment HNSCC surveillance imaging

Practical imaging informatics : foundations and applications for medical imaging

This new edition is a comprehensive source of imaging informatics fundamentals and how those fundamentals are applied in everyday practice. Imaging Informatics Professionals (IIPs) play a critical role in healthcare, and the scope of the profession has grown far beyond the boundaries of the PACS. A successful IIP must understand the PACS itself and all the software systems networked together in the medical environment. Additionally, an IIP must know the workflows of all the imaging team members, have a base in several medical specialties and be fully capable in the realm of information technology. Practical Imaging Informatics has been reorganized to follow a logical progression from basic background information on IT and clinical image management, through daily operations and troubleshooting, to long-term planning. The book has been fully updated to include the latest technologies and procedures, including artificial intelligence and machine learning. Written by a team of renowned international authors from the Society for Imaging Informatics in Medicine and the European Society of Medical Imaging Informatics, this book is an indispensable reference for the practicing IIP. In addition, it is an ideal guide for those studying for a certification exam, biomedical informaticians, trainees with an interest in informatics, and any professional who needs quick access to the nuts and bolts of imaging informatics

Evidence for an adverse impact of remote readouts on radiology resident productivity: Implications for training and clinical practice.

After their rapid adoption at the onset of the coronavirus pandemic, remote case reviews (remote readouts) between diagnostic radiology residents and their attendings have persisted in an increasingly remote workforce, despite relaxing social distancing guidelines. Our objective was to evaluate the impact of the transition to remote readouts on resident case volumes after the recovery of institutional volumes. We tabulated radiology reports co-authored by first-to-third-year radiology residents (R1-R3) between July 1 and December 31 of the first pandemic year, 2020, and compared to the prior two pre-pandemic years. Half-years were analyzed because institutional volumes recovered by July 2020. Resident volumes were normalized to rotations, which were in divisions categorized by the location of the supervising faculty during the pandemic period; in 'remote' divisions, all faculty worked off-site, whereas 'hybrid' divisions had a mix of attendings working on-site and remotely. All residents worked on-site. Data analysis was performed with Student's t test and multivariate linear regression. The largest drops in total case volume occurred in the two remote divisions (38% [6,086 to 3,788], and 26% [11,046 to 8,149]). None of the hybrid divisions with both in-person and remote supervision decreased by more than 5%. With multivariate regression, a resident assigned to a standardized remote rotation in 2020 would complete 32% (253 to 172) fewer studies than in identical pre-pandemic rotations (coefficent of -81.6, p = .005) but would be similar for hybrid rotations. R1 residents would be expected to interpret 40% fewer (180 to 108) cases on remote rotations during the pandemic (coefficient of -72.3, p = .007). No significant effect was seen for R2 or R3 residents (p = .099 and p = .29, respectively). Radiology residents interpreted fewer studies during remote rotations than on hybrid rotations that included in-person readouts. As resident case volume is correlated with clinical performance and board pass rate, monitoring the readout model for downstream educational effects is essential. Until evidence shows that educational outcomes remain unchanged, radiology residencies may wish to preserve in-person resident readouts, particularly for junior residents

Characterization of the number of residents and resident co-authored studies.

Characterization of the number of residents and resident co-authored studies.</p

Multivariate regression by resident level.

† Cross terms are denoted with • between interacting variables. p values of st and December 31st, 2020. FL: Fluoro; MG: Mammography; XR: Radiographs; US: Ultrasound, NM: Nuclear Medicine, CT/MR: Cross-sectional studies. (DOCX)</p

Mean number of co-authored reports by each resident per rotation.

Mean number of co-authored reports by each resident per rotation.</p

Mean number of resident co-authored studies per rotation, by readout model and resident year.

Pre-pandemic encompasses July 1st through December 31st 2018 and 2019 and is compared to the pandemic period during July 1st through December 31st of 2020. Two-sided t-test, * denotes statistically significant p values (DOCX)</p

The effect of the transition to a remote readout model on resident case volumes.

A model of multivariate regression analysis controlling for image modality was used to calculate the marginal effects of the readout model on resident case volumes before or during the pandemic. Results are shown for all residents and subdivided by resident level (R1-R3). The pre-pandemic period (shown in grey) encompasses the latter half of 2018 and 2019 and is compared to the pandemic period during latter half of 2020. p values < .05 and < .01 are denoted by * and **, respectively; error bars represent CI for pandemic values.</p

Resident case volumes before and during the pandemic.

Readout models were adopted for all faculty in remote divisions and for a proportion of faculty in hybrid divisions during the pandemic period (grey column, July 1 to December 31, 2020). Resident co-authored reports were compared to pre-pandemic data (black column; July 1 to December 31, 2018, 2019). A) Average case volumes were normalized to rotation for all the residents (R1-R3, N = 56) and to B) resident by level.</p