Accurate and Reliable Classification of Unstructured Reports on Their Diagnostic Goal Using BERT Models

Understanding the diagnostic goal of medical reports is valuable information for understanding patient flows. This work focuses on extracting the reason for taking an MRI scan of Multiple Sclerosis (MS) patients using the attached free-form reports: Diagnosis, Progression or Monitoring. We investigate the performance of domain-dependent and general state-of-the-art language models and their alignment with domain expertise. To this end, eXplainable Artificial Intelligence (XAI) techniques are used to acquire insight into the inner workings of the model, which are verified on their trustworthiness. The verified XAI explanations are then compared with explanations from a domain expert, to indirectly determine the reliability of the model. BERTje, a Dutch Bidirectional Encoder Representations from Transformers (BERT) model, outperforms RobBERT and MedRoBERTa.nl in both accuracy and reliability. The latter model (MedRoBERTa.nl) is a domain-specific model, while BERTje is a generic model, showing that domain-specific models are not always superior. Our validation of BERTje in a small prospective study shows promising results for the potential uptake of the model in a practical setting.</p

Accurate and Reliable Classification of Unstructured Reports on Their Diagnostic Goal Using BERT Models

Understanding the diagnostic goal of medical reports is valuable information for understanding patient flows. This work focuses on extracting the reason for taking an MRI scan of Multiple Sclerosis (MS) patients using the attached free-form reports: Diagnosis, Progression or Monitoring. We investigate the performance of domain-dependent and general state-of-the-art language models and their alignment with domain expertise. To this end, eXplainable Artificial Intelligence (XAI) techniques are used to acquire insight into the inner workings of the model, which are verified on their trustworthiness. The verified XAI explanations are then compared with explanations from a domain expert, to indirectly determine the reliability of the model. BERTje, a Dutch Bidirectional Encoder Representations from Transformers (BERT) model, outperforms RobBERT and MedRoBERTa.nl in both accuracy and reliability. The latter model (MedRoBERTa.nl) is a domain-specific model, while BERTje is a generic model, showing that domain-specific models are not always superior. Our validation of BERTje in a small prospective study shows promising results for the potential uptake of the model in a practical setting.</p

Weakly Supervised Learning for Breast Cancer Prediction on Mammograms in Realistic Settings

Automatic methods for early detection of breast cancer on mammography can significantly decrease mortality. Broad uptake of those methods in hospitals is currently hindered because the methods have too many constraints. They assume annotations available for single images or even regions-of-interest (ROIs), and a fixed number of images per patient. Both assumptions do not hold in a general hospital setting. Relaxing those assumptions results in a weakly supervised learning setting, where labels are available per case, but not for individual images or ROIs. Not all images taken for a patient contain malignant regions and the malignant ROIs cover only a tiny part of an image, whereas most image regions represent benign tissue. In this work, we investigate a two-level multi-instance learning (MIL) approach for case-level breast cancer prediction on two public datasets (1.6k and 5k cases) and an in-house dataset of 21k cases. Observing that breast cancer is usually only present in one side, while images of both breasts are taken as a precaution, we propose a domain-specific MIL pooling variant. We show that two-level MIL can be applied in realistic clinical settings where only case labels, and a variable number of images per patient are available. Data in realistic settings scales with continuous patient intake, while manual annotation efforts do not. Hence, research should focus in particular on unsupervised ROI extraction, in order to improve breast cancer prediction for all patients.Comment: 10 pages, 5 figures, 5 table

Weakly Supervised Learning for Breast Cancer Prediction on Mammograms in Realistic Settings

Automatic methods for early detection of breast cancer on mammography can significantly decrease mortality. Broad uptake of those methods in hospitals is currently hindered because the methods have too many constraints. They assume annotations available for single images or even regions-of-interest (ROIs), and a fixed number of images per patient. Both assumptions do not hold in a general hospital setting. Relaxing those assumptions results in a weakly supervised learning setting, where labels are available per case, but not for individual images or ROIs. Not all images taken for a patient contain malignant regions and the malignant ROIs cover only a tiny part of an image, whereas most image regions represent benign tissue. In this work, we investigate a two-level multi-instance learning (MIL) approach for case-level breast cancer prediction on two public datasets (1.6k and 5k cases) and an in-house dataset of 21k cases. Observing that breast cancer is usually only present in one side, while images of both breasts are taken as a precaution, we propose a domain-specific MIL pooling variant. We show that two-level MIL can be applied in realistic clinical settings where only case labels, and a variable number of images per patient are available. Data in realistic settings scales with continuous patient intake, while manual annotation efforts do not. Hence, research should focus in particular on unsupervised ROI extraction, in order to improve breast cancer prediction for all patients

A direct comparison in diagnostic performance of CDUS, FDG-PET/CT and MRI in patients suspected of giant cell arteritis

Objectives: This study directly compares the diagnostic performance of colour duplex ultrasound (CDUS), fluor-18-deoxyglucose positron emission tomography computed tomography (FDG-PET/CT) and magnetic resonance imaging (MRI) in patients suspected of giant cell arteritis (GCA).Methods: Patients with suspected GCA were included in a nested-case control pilot study. CDUS, whole body FDG-PET/CT and cranial MRI were performed within 5 working days after initial clinical evaluation. Clinical diagnosis after six months follow-up by experienced rheumatologists in the field of GCA, blinded for imaging, was used as reference standard. Diagnostic performance of the imaging modalities was determined. Stratification for GCA subtype was performed and imaging results were evaluated in different risk stratification groups.Results: In total, 23 patients with GCA and 19 patients suspected of but not diagnosed with GCA were included. Sensitivity was 69.6% (95%CI 50.4%–88.8%) for CDUS, 52.2% (95%CI 31.4%–73.0%) for FDG-PET/CT and 56.5% (95%CI 35.8%–77.2%) for MRI. Specificity was 100% for CDUS, FDG-PET/CT and MRI. FDG-PET/CT was negative for GCA in all isolated cranial GCA patients (n = 8), while MRI was negative in all isolated extracranial GCA patients (n = 4). In four GCA patients with false-negative (n = 2; intermediate and high risk) or inconclusive (n = 2; low and intermediate risk) CDUS results, further imaging confirmed diagnosis.Conclusions: Sensitivity of CDUS was highest, while specificity was excellent in all imaging modalities. Nevertheless, confidence intervals of all imaging modalities were overlapping. Following EULAR recommendations, CDUS can be used as a first test to diagnose GCA. With insufficient evidence for GCA, further testing considering GCA subtype is warranted

A Tale of Diagnostic Delay with Detrimental Consequences: Illustrating the Challenging Nature of Diagnosing Giant Cell Arteritis

Giant cell arteritis is a medical emergency as severe, irreversible complications may occur if it is not treated in a timely manner. However, in daily practice early diagnosis can be challenging. We report the case of a 70-year-old woman who presented with multiple ischaemic cerebral vascular accidents related to newly diagnosed giant cell arteritis. Review of her charts revealed a substantial delay from the onset of symptoms to diagnosis. This case demonstrates the need for additional efforts to reduce delay in referring patients with giant cell arteritis and the need to implement fast-track clinics to prevent serious complications.</jats:p

Accurate and Reliable Classification of Unstructured Reports on Their Diagnostic Goal Using BERT Models

Understanding the diagnostic goal of medical reports is valuable information for understanding patient flows. This work focuses on extracting the reason for taking an MRI scan of Multiple Sclerosis (MS) patients using the attached free-form reports: Diagnosis, Progression or Monitoring. We investigate the performance of domain-dependent and general state-of-the-art language models and their alignment with domain expertise. To this end, eXplainable Artificial Intelligence (XAI) techniques are used to acquire insight into the inner workings of the model, which are verified on their trustworthiness. The verified XAI explanations are then compared with explanations from a domain expert, to indirectly determine the reliability of the model. BERTje, a Dutch Bidirectional Encoder Representations from Transformers (BERT) model, outperforms RobBERT and MedRoBERTa.nl in both accuracy and reliability. The latter model (MedRoBERTa.nl) is a domain-specific model, while BERTje is a generic model, showing that domain-specific models are not always superior. Our validation of BERTje in a small prospective study shows promising results for the potential uptake of the model in a practical setting.</jats:p