EndoNet: model for automatic calculation of H-score on histological slides

H-score is a semi-quantitative method used to assess the presence and distribution of proteins in tissue samples by combining the intensity of staining and percentage of stained nuclei. It is widely used but time-consuming and can be limited in accuracy and precision. Computer-aided methods may help overcome these limitations and improve the efficiency of pathologists' workflows. In this work, we developed a model EndoNet for automatic calculation of H-score on histological slides. Our proposed method uses neural networks and consists of two main parts. The first is a detection model which predicts keypoints of centers of nuclei. The second is a H-score module which calculates the value of the H-score using mean pixel values of predicted keypoints. Our model was trained and validated on 1780 annotated tiles with a shape of 100x100 $\mu m$ and performed 0.77 mAP on a test dataset. Moreover, the model can be adjusted to a specific specialist or whole laboratory to reproduce the manner of calculating the H-score. Thus, EndoNet is effective and robust in the analysis of histology slides, which can improve and significantly accelerate the work of pathologists

Explainable artificial intelligence (XAI) in deep learning-based medical image analysis

With an increase in deep learning-based methods, the call for explainability of such methods grows, especially in high-stakes decision making areas such as medical image analysis. This survey presents an overview of eXplainable Artificial Intelligence (XAI) used in deep learning-based medical image analysis. A framework of XAI criteria is introduced to classify deep learning-based medical image analysis methods. Papers on XAI techniques in medical image analysis are then surveyed and categorized according to the framework and according to anatomical location. The paper concludes with an outlook of future opportunities for XAI in medical image analysis.Comment: Submitted for publication. Comments welcome by email to first autho

A survey of deep learning models in medical therapeutic areas.

pre-print1407 K

Artificial intelligence in digital pathology: a diagnostic test accuracy systematic review and meta-analysis

Ensuring diagnostic performance of AI models before clinical use is key to the safe and successful adoption of these technologies. Studies reporting AI applied to digital pathology images for diagnostic purposes have rapidly increased in number in recent years. The aim of this work is to provide an overview of the diagnostic accuracy of AI in digital pathology images from all areas of pathology. This systematic review and meta-analysis included diagnostic accuracy studies using any type of artificial intelligence applied to whole slide images (WSIs) in any disease type. The reference standard was diagnosis through histopathological assessment and / or immunohistochemistry. Searches were conducted in PubMed, EMBASE and CENTRAL in June 2022. We identified 2976 studies, of which 100 were included in the review and 48 in the full meta-analysis. Risk of bias and concerns of applicability were assessed using the QUADAS-2 tool. Data extraction was conducted by two investigators and meta-analysis was performed using a bivariate random effects model. 100 studies were identified for inclusion, equating to over 152,000 whole slide images (WSIs) and representing many disease types. Of these, 48 studies were included in the meta-analysis. These studies reported a mean sensitivity of 96.3% (CI 94.1-97.7) and mean specificity of 93.3% (CI 90.5-95.4) for AI. There was substantial heterogeneity in study design and all 100 studies identified for inclusion had at least one area at high or unclear risk of bias. This review provides a broad overview of AI performance across applications in whole slide imaging. However, there is huge variability in study design and available performance data, with details around the conduct of the study and make up of the datasets frequently missing. Overall, AI offers good accuracy when applied to WSIs but requires more rigorous evaluation of its performance.Comment: 26 pages, 5 figures, 8 tables + Supplementary material

Assessment of Digital Pathology Imaging Biomarkers Associated with Breast Cancer Histologic Grade

Background: Evaluating histologic grade for breast cancer diagnosis is standard and associated with prognostic outcomes. Current challenges include the time required for manual microscopic evaluation and interobserver variability. This study proposes a computer-aided diagnostic (CAD) pipeline for grading tumors using artificial intelligence. Methods: There were 138 patients included in this retrospective study. Breast core biopsy slides were prepared using standard laboratory techniques, digitized, and pre-processed for analysis. Deep convolutional neural networks (CNNs) were developed to identify the regions of interest containing malignant cells and to segment tumor nuclei. Imaging-based features associated with spatial parameters were extracted from the segmented regions of interest (ROIs). Clinical datasets and pathologic biomarkers (estrogen receptor, progesterone receptor, and human epidermal growth factor 2) were collected from all study subjects. Pathologic, clinical, and imaging-based features were input into machine learning (ML) models to classify histologic grade, and model performances were tested against ground-truth labels at the patient-level. Classification performances were evaluated using receiver-operating characteristic (ROC) analysis. Results: Multiparametric feature sets, containing both clinical and imaging-based features, demonstrated high classification performance. Using imaging-derived markers alone, the classification performance demonstrated an area under the curve (AUC) of 0.745, while modeling these features with other pathologic biomarkers yielded an AUC of 0.836. Conclusion: These results demonstrate an association between tumor nuclear spatial features and tumor grade. If further validated, these systems may be implemented into pathology CADs and can assist pathologists to expeditiously grade tumors at the time of diagnosis and to help guide clinical decisions

KimiaNet: Training a Deep Network for Histopathology using High-Cellularity

With the recent progress in deep learning, one of the common approaches to represent images is extracting deep features. A primitive way to do this is by using off-the-shelf models. However, these features could be improved through fine-tuning or even training a network from scratch by domain-specific images. This desirable task is hindered by the lack of annotated or labeled images in the field of histopathology. In this thesis, a new network, namely KimiaNet, is proposed that uses an existing dense topology but is tailored for generating informative and discriminative deep features from histopathology images for image representation. This model is trained based on the existing DenseNet-121 architecture but by using more than 240,000 image patches of 1000 ⨉ 1000 pixels acquired at 20⨉ magnification. Considering the high cost of histopathology image annotation, which makes the idea impractical at a large scale, a high-cellularity mosaic approach is suggested which could be used as a weak or soft labeling method. Patches used for training the KimiaNet are extracted from 7,126 whole slide images of formalin-fixed paraffin-embedded (FFPE) biopsy samples, spanning 30 cancer sub-types and publicly available through The Cancer Genome Atlas (TCGA) repository. The quality of features generated by KimiaNet are tested via two types of image search, (i) given a query slide, searching among all of the slides and finding the ones with the tissue type similar to the query’s and (ii) searching among slides within the query slide’s tumor type and finding slides with the same cancer sub-type as the query slide’s. Compared to the pre-trained DenseNet-121 and the fine-tuned versions, KimiaNet achieved predominantly the best results for both search modes. In order to get an intuition of how effective training from scratch is on the expressiveness of the deep features, the deep features of randomly selected patches, from each cancer subtype, are extracted using both KimiaNet and pre-trained DenseNet-121 and visualized after reducing their dimensionality using t-distributed Stochastic Neighbor Embedding (tSNE). This visualization illustrates that for KimiaNet, the instances of each class can easily be distinguished from others while for pre-trained DenseNet the instances of almost all of the classes are mixed together. This comparison is another verification to show that how discriminative training with domain-specific images has made the features. Also, four simpler networks, made up of repetitions of convolutional, batch-normalization and Rectified Linear Unit (ReLU) layers, (CBR networks) are implemented and compared against the KimiaNet to check if the network design could still be further simplified. The experiments demonstrated that KimiaNet features are by far better than CBR networks which validate the DenseNet-121 as a good candidate for KimiaNet’s architecture

Endometriosis

Endometriosis is a common and serious disease that is estimated to cost the world economy $9.7 billion a year. Most of these costs come from lost productivity at work. As such, it is important to help women receive earlier diagnosis and more effective treatment. This book presents a comprehensive overview of endometriosis, including information on molecular diagnostics and imaging methods for early detection as well as new, less-invasive treatments that preserve women’s fertility

Detection and staging of colonic lesions using computed tomography and magnetic resonance imaging

In Sweden, more than 6000 new patients were diagnosed with colorectal cancer in 2015 of which over 4000 patients had colon cancer and 1800 died from the disease. It is the second most common cancer after breast- and prostate cancer. In the last decade, significant improvement in the treatment of both rectal and colon cancer have been achieved. Diagnostic imaging, using CT, MRI and PET/CT, has become essential in the preoperative work-up. New neoadjuvant treatment strategies are under study in colon cancer. In the selection of patients for these treatments, pre- and post-treatment imaging has also become of interest. The overall aim of this thesis is to evaluate cross sectional imaging modalities for detection of colonic polyps and staging of patients with colon cancer using CT and MRI. The aim of paper 1 was to investigate the impact of radiation dose and spatial resolution in detecting colonic polyps in a phantom study simulating computed tomographic colonography (CTC). By using different scanning protocols with different slice -thickness, pitch and tube current we showed that the dose level could be substantially reduced by lowering the tube current without compromising the detection rate for polyps larger than 5 mm. The aim of paper 2 was to evaluate if high resolution MRI of colon cancer contributed to the standard staging procedure with CT with respect to assessment of local tumour extent, nodal staging and extramural venous invasion (EMVI). An advantage of MRI over CT due to its soft tissue discrimination to identify prognostic factors such as tumour stage and extramural venous invasion was found. The result of nodal staging for both modalities were equally moderate. The aim of paper 3 was to evaluate commonly used imaging CT criteria for lymph node metastases in predicting stage III disease. Of the different imaging criteria, morphological features performed best specifically internal heterogeneity and irregular outer border. None of the size criteria were predictive. The aim of paper 4 was to validate morphological CT criteria from paper 3 in a prospectively collected patient cohort using two observers. By using the criteria internal heterogeneity and a combination including irregular outer border, a moderate sensitivity and high specificity was achieved predicting stage III disease. CT and high resolution MRI can be used to classify colonic tumours into not locally advanced or locally advanced. The prediction of lymph node metastases with the most commonly used image modalities is however unsettled and challenged. In the setting of selecting patients to neoadjuvant chemotherapy, the ongoing trials so far have used inclusion criteria based on tumour T-stage only (T3cd-T4 as locally advanced). Patients with lower tumour T-stage but still have other adverse prognostic feature such as regional metastases will therefore potentially be undertreated and patients with no metastases will potentially be overtreated. Search for other prognostic factors identified on cross sectional imaging has to be performed