Automated detection and delineation of lymph nodes in haematoxylin & eosin stained digitised slides

Treatment of patients with oesophageal and gastric cancer (OeGC) is guided by disease stage, patient performance status and preferences. Lymph node (LN) status is one of the strongest prognostic factors for OeGC patients. However, survival varies between patients with the same disease stage and LN status. We recently showed that LN size from patients with OeGC might also have prognostic value, thus making delineations of LNs essential for size estimation and the extraction of other imaging biomarkers. We hypothesized that a machine learning workflow is able to: (1) find digital H&E stained slides containing LNs, (2) create a scoring system providing degrees of certainty for the results, and (3) delineate LNs in those images. To train and validate the pipeline, we used 1695 H&E slides from the OE02 trial. The dataset was divided into training (80%) and validation (20%). The model was tested on an external dataset of 826 H&E slides from the OE05 trial. U-Net architecture was used to generate prediction maps from which predefined features were extracted. These features were subsequently used to train an XGBoost model to determine if a region truly contained a LN. With our innovative method, the balanced accuracies of the LN detection were 0.93 on the validation dataset (0.83 on the test dataset) compared to 0.81 (0.81) on the validation (test) datasets when using the standard method of thresholding U-Net predictions to arrive at a binary mask. Our method allowed for the creation of an “uncertain” category, and partly limited false-positive predictions on the external dataset. The mean Dice score was 0.73 (0.60) per-image and 0.66 (0.48) per-LN for the validation (test) datasets. Our pipeline detects images with LNs more accurately than conventional methods, and high-throughput delineation of LNs can facilitate future LN content analyses of large datasets

Automated detection and delineation of lymph nodes in haematoxylin & eosin stained digitised slides.

Treatment of patients with oesophageal and gastric cancer (OeGC) is guided by disease stage, patient performance status and preferences. Lymph node (LN) status is one of the strongest prognostic factors for OeGC patients. However, survival varies between patients with the same disease stage and LN status. We recently showed that LN size from patients with OeGC might also have prognostic value, thus making delineations of LNs essential for size estimation and the extraction of other imaging biomarkers. We hypothesized that a machine learning workflow is able to: (1) find digital H&E stained slides containing LNs, (2) create a scoring system providing degrees of certainty for the results, and (3) delineate LNs in those images. To train and validate the pipeline, we used 1695 H&E slides from the OE02 trial. The dataset was divided into training (80%) and validation (20%). The model was tested on an external dataset of 826 H&E slides from the OE05 trial. U-Net architecture was used to generate prediction maps from which predefined features were extracted. These features were subsequently used to train an XGBoost model to determine if a region truly contained a LN. With our innovative method, the balanced accuracies of the LN detection were 0.93 on the validation dataset (0.83 on the test dataset) compared to 0.81 (0.81) on the validation (test) datasets when using the standard method of thresholding U-Net predictions to arrive at a binary mask. Our method allowed for the creation of an "uncertain" category, and partly limited false-positive predictions on the external dataset. The mean Dice score was 0.73 (0.60) per-image and 0.66 (0.48) per-LN for the validation (test) datasets. Our pipeline detects images with LNs more accurately than conventional methods, and high-throughput delineation of LNs can facilitate future LN content analyses of large datasets

The creation of a large set of realistic synthetic microcalcification clusters for simulation in (contrast-enhanced) mammography images

Characterization of microcalcification clusters in the breast and differentiation between benign and malignant structures on (contrast-enhanced) mammography (CEM) images is of great importance to determine cancerous lesions. Computer algorithms may help performing these tasks, but typically need large sets of data for model training. Therefore this paper develops a method to create synthetic microcalcification clusters that can later be used to overcome data sparsity problems. Starting from descriptors of the shape and size, both benign and malignant microcalcifications were created and then combined into 3-dimensional cluster models given realistic geometric properties. The distributions of the largest diameter and the number of microcalcifications per cluster in a set of 500 simulated clusters were set such that they agreed with those of real clusters. An existing simulation tool was then extended to insert the clusters into processed, low-energy CEM background images with appropriate contrast values. In a validation study comprised of 40 real and 40 synthetic cases, radiologists were asked to evaluate realism and malignancy. It was found that the shape and the structure of the individual microcalcifications as well as the complete clusters were realistic. Thus the descriptors were chosen correctly and enabled a good classification between benign and malignant cases. The realistic brightness and boundary smoothness proved the simulation tool can correctly insert the 3D clusters into real background images and is suitable of creating a large set of realistic microcalcification clusters simulated in existing (contrast-enhanced) mammography images. With improvements on the correspondence of insertion location in craniocaudal and mediolateral oblique view, which proved more challenging to simulate realistically, this promising method is expected to be applicable for modeling complete synthetic cases. Such a dataset can be used for data enrichment where data sources are limited and for development and training purposes

Machine learning for grading and prognosis of esophageal dysplasia using mass spectrometry and histological imaging

BACKGROUND: Barrett's esophagus (BE) is a precursor lesion of esophageal adenocarcinoma and may progress from non-dysplastic through low-grade dysplasia (LGD) to high-grade dysplasia (HGD) and cancer. Grading BE is of crucial prognostic value and is currently based on the subjective evaluation of biopsies. This study aims to investigate the potential of machine learning (ML) using spatially resolved molecular data from mass spectrometry imaging (MSI) and histological data from microscopic hematoxylin and eosin (H&E)-stained imaging for computer-aided diagnosis and prognosis of BE. METHODS: Biopsies from 57 patients were considered, divided into non-dysplastic (n = 15), LGD non-progressive (n = 14), LGD progressive (n = 14), and HGD (n = 14). MSI experiments were conducted at 50 × 50 μm spatial resolution per pixel corresponding to a tile size of 96x96 pixels in the co-registered H&E images, making a total of 144,823 tiles for the whole dataset. RESULTS: ML models were trained to distinguish epithelial tissue from stroma with area-under-the-curve (AUC) values of 0.89 (MSI) and 0.95 (H&E)) and dysplastic grade (AUC of 0.97 (MSI) and 0.85 (H&E)) on a tile level, and low-grade progressors from non-progressors on a patient level (accuracies of 0.72 (MSI) and 0.48 (H&E)). CONCLUSIONS: In summary, while the H&E-based classifier was best at distinguishing tissue types, the MSI-based model was more accurate at distinguishing dysplastic grades and patients at progression risk, which demonstrates the complementarity of both approaches. Data are available via ProteomeXchange with identifier PXD028949

Radiomics for precision medicine: Current challenges, future prospects, and the proposal of a new framework

peer reviewedThe advancement of artificial intelligence concurrent with the development of medical imaging techniques provided a unique opportunity to turn medical imaging from mostly qualitative, to further quantitative and mineable data that can be explored for the development of clinical decision support systems (cDSS). Radiomics, a method for the high throughput extraction of hand-crafted features from medical images, and deep learning -the data driven modeling techniques based on the principles of simplified brain neuron interactions, are the most researched quantitative imaging techniques. Many studies reported on the potential of such techniques in the context of cDSS. Such techniques could be highly appealing due to the reuse of existing data, automation of clinical workflows, minimal invasiveness, three-dimensional volumetric characterization, and the promise of high accuracy and reproducibility of results and cost-effectiveness. Nevertheless, there are several challenges that quantitative imaging techniques face, and need to be addressed before the translation to clinical use. These challenges include, but are not limited to, the explainability of the models, the reproducibility of the quantitative imaging features, and their sensitivity to variations in image acquisition and reconstruction parameters. In this narrative review, we report on the status of quantitative medical image analysis using radiomics and deep learning, the challenges the field is facing, propose a framework for robust radiomics analysis, and discuss future prospects

Radiomics for precision medicine: Current challenges, future prospects, and the proposal of a new framework

The advancement of artificial intelligence concurrent with the development of medical imaging techniques provided a unique opportunity to turn medical imaging from mostly qualitative, to further quantitative and mineable data that can be explored for the development of clinical decision support systems (cDSS). Radiomics, a method for the high throughput extraction of hand-crafted features from medical images, and deep learning-the data driven modeling techniques based on the principles of simplified brain neuron interactions, are the most researched quantitative imaging techniques. Many studies reported on the potential of such techniques in the context of cDSS. Such techniques could be highly appealing due to the reuse of existing data, automation of clinical workflows, minimal invasiveness, three-dimensional volumetric characterization, and the promise of high accuracy and reproducibility of results and cost-effectiveness. Nevertheless, there are several challenges that quantitative imaging techniques face, and need to be addressed before the translation to clinical use. These challenges include, but are not limited to, the explainability of the models, the reproducibility of the quantitative imaging features, and their sensitivity to variations in image acquisition and reconstruction parameters. In this narrative review, we report on the status of quantitative medical image analysis using radiomics and deep learning, the challenges the field is facing, propose a framework for robust radiomics analysis, and discuss future prospects

Towards texture accurate slice interpolation of medical images using PixelMiner

peer reviewedQuantitative image analysis models are used for medical imaging tasks such as registration, classification, object detection, and segmentation. For these models to be capable of making accurate predictions, they need valid and precise information. We propose PixelMiner, a convolution-based deep-learning model for interpolating computed tomography (CT) imaging slices. PixelMiner was designed to produce texture-accurate slice interpolations by trading off pixel accuracy for texture accuracy. PixelMiner was trained on a dataset of 7829 CT scans and validated using an external dataset. We demonstrated the model's effectiveness by using the structural similarity index (SSIM), peak signal to noise ratio (PSNR), and the root mean squared error (RMSE) of extracted texture features. Additionally, we developed and used a new metric, the mean squared mapped feature error (MSMFE). The performance of PixelMiner was compared to four other interpolation methods: (tri-)linear, (tri-)cubic, windowed sinc (WS), and nearest neighbor (NN). PixelMiner produced texture with a significantly lowest average texture error compared to all other methods with a normalized root mean squared error (NRMSE) of 0.11 (p < .01), and the significantly highest reproducibility with a concordance correlation coefficient (CCC) ≥ 0.85 (p < .01). PixelMiner was not only shown to better preserve features but was also validated using an ablation study by removing auto-regression from the model and was shown to improve segmentations on interpolated slices

Automated detection and segmentation of non-small cell lung cancer computed tomography images

Correct interpretation of computer tomography (CT) scans is important for the correct assessment of a patient's disease but can be subjective and timely. Here, the authors develop a system that can automatically segment the non-small cell lung cancer on CT images of patients and show in an in silico trial that the method was faster and more reproducible than clinicians.Detection and segmentation of abnormalities on medical images is highly important for patient management including diagnosis, radiotherapy, response evaluation, as well as for quantitative image research. We present a fully automated pipeline for the detection and volumetric segmentation of non-small cell lung cancer (NSCLC) developed and validated on 1328 thoracic CT scans from 8 institutions. Along with quantitative performance detailed by image slice thickness, tumor size, image interpretation difficulty, and tumor location, we report an in-silico prospective clinical trial, where we show that the proposed method is faster and more reproducible compared to the experts. Moreover, we demonstrate that on average, radiologists & radiation oncologists preferred automatic segmentations in 56% of the cases. Additionally, we evaluate the prognostic power of the automatic contours by applying RECIST criteria and measuring the tumor volumes. Segmentations by our method stratified patients into low and high survival groups with higher significance compared to those methods based on manual contours