Recognizing Microscopic Structures: Dense Semantic Segmentation of Multiple Histopathological Classes using Fully Convolutional Neural Networks

In order to alleviate the financial burden on the healthcare sector as well as relax its employees’ workload, there is a need to introduce novel tools that automate some of the tasks that today are performed manually. Especially pathology poses a problem with few pathologists, demanding manual labour and unnecessary work on benign tissue. As a response, the DOGS project aims to develop a tool to automate or assist in Gleason grading of histopathological images from prostate biopsies. It is probable that such a tool would benefit from having access to individually segmented, pathologically relevant objects from the images. Moreover, considering recent advances in deep learning and its frequently impressive performance on various image analysis tasks, it is natural to approach this challenge from a deep learning perspective. This thesis proposes several fully convolutional neural networks to be used for dense semantic segmentation on histopathological images. The networks’ architectures are all initially based on already proven networks but are modified in various ways to achieve better performance. Being a supervised machine learning task, the ground truth required to train the network has been developed as a part of the thesis. The best-performing network obtained an accuracy of 79.71 % mean intersection over union and the networks presented plausibly equaled or outperformed state-of-the-art methods in nuclei segmentation. However, further work is deemed necessary for reaching adequate segmentation performance. Several suggestions for possible future directions of work are presented, as well as obstacles that have to be considered moving onwards.To make a significant dent in the issues the healthcare sector faces today in terms of costs and overextended employees, a great increase of viable automated tools will sooner or later be needed. For pathologists this is no different. However, the complexity and size of microscopy images makes automated analysis of them difficult. This thesis achieved promising and first-of-its-kind results in tackling the very underexplored challenge of recognizing several structures in microscopy images at once. By using a deep learning approach heavily inspired from a pair of popular artificial neural networks a score of 80 % mean IU was reached. The resulting networks are prospects for use in several different preprocessing steps in medical image analysis applications – possibly enabling or improving automated tools in the pathological field in the future

Applications of Deep Learning in Medical Image Analysis : Grading of Prostate Cancer and Detection of Coronary Artery Disease

A wide range of medical examinations are using analysis of images from different types of equipment. Using artificial intelligence, the assessments could be done automatically. This can have multiple benefits for the healthcare; reduce workload for medical doctors, decrease variations in diagnoses and cut waiting times for the patient as well as improve the performance. The aim of this thesis has been to develop such solutions for two common diseases: prostate cancer and coronary artery disease. The methods used are mainly based on deep learning, where the model teaches itself by training on large datasets.Prostate cancer is one of the most common cancer diagnoses among men. The diagnosis is most commonly determined by visual assessment of prostate biopsies in a light microscope according to the Gleason scale. Deep learning methods to automatically detect and grade the cancer areas are presented in this thesis. The methods have been adapted to improve the generalisation performance on images from different hospitals, images which have inevitable variations in e.g.\ stain appearance. The methods include the usage of digital stain normalisation, training with extensive augmentation or using models such as a domain-adversarial neural network. One Gleason grading algorithm was evaluated on a small cohort with biopsies annotated in detail by two pathologists, to compare the performance with pathologists' inter-observer variability. Another cancer detection algorithm was evaluated on a large active surveillance cohort, containing patients with small areas of low-grade cancer. The results are promising towards a future tool to facilitate grading of prostate cancer.Cardiovascular disease is the leading cause of death world-wide, whereof coronary artery disease is one of the most common diseases. One way to diagnose coronary artery disease is by using myocardial perfusion imaging, where disease in the three main arteries supplying the heart with blood can be detected. Methods based on deep learning to perform the detection automatically are presented in this thesis. Furthermore, an algorithm developed to predict the degree of coronary artery stenosis from myocardial perfusion imaging, by means of quantitative coronary angiography, has also been developed. This assessment is normally done using invasive coronary angiography. Making the prediction automatically from myocardial perfusion imaging could save suffering for patients and free resources within the healthcare system

Artificial intelligence for streamlining prostate cancer diagnostics

With around 1.2 million cases per year, prostate cancer is the second most common cancer among men. It is usually a slow growing disease that affects older men. It is also a cancer that is heterogenous, often multifocal, and rarely show symptoms as long as it is localized. All these things make the disease difficult to detect, diagnose and study. The objective of this thesis is to develop and improve technologies for prostate cancer diagnostics and to acquire knowledge related to these technologies that directly translate to clinical utility. In Study I, we extended analysis of the multivariable diagnostic prediction model S3M by exploring the relative contribution from the individual predictors and evaluating the model in reflex setting where the test is only given to men positive on a PSA test. We also updated the list of included predictors and refitted the model to more data. In Study II, we digitized a substantial part of the biopsy cores collected from the men in study I. These images were used to develop and validate an AI for prostate cancer diagnostics by detecting, grading, and measuring the extent of cancer in the biopsies. The AI achieved nearly perfect detection of cancer and expert pathologist level grading of the biopsies. It also well predicted the total tumor burden of the patient. In Study III, we focused our attention on perineural invasion, a common finding in prostate biopsies. This study has added to the evidence that there is substantial and independent prognostic information in this finding and argued that it should be included as a compulsory part in pathology reporting guidelines for prostate biopsies. In Study IV, we developed an AI for detection and localization of perineural invasion in biopsies. The AI achieved high discriminative ability on an independent test set. We are currently collecting external data to validate these results in another environment and to compare the results of the AI against expert pathologists. In conclusion, the technologies developed in this thesis has shown promise in streamlining the clinical workload around prostate cancer detection and diagnostics. The thesis has also contributed to pieces of information related to these technologies

Improving Prostate Cancer Detection with Breast Histopathology Images

Deep neural networks have introduced significant advancements in the field of machine learning-based analysis of digital pathology images including prostate tissue images. With the help of transfer learning, classification and segmentation performance of neural network models have been further increased. However, due to the absence of large, extensively annotated, publicly available prostate histopathology datasets, several previous studies employ datasets from well-studied computer vision tasks such as ImageNet dataset. In this work, we propose a transfer learning scheme from breast histopathology images to improve prostate cancer detection performance. We validate our approach on annotated prostate whole slide images by using a publicly available breast histopathology dataset as pre-training. We show that the proposed cross-cancer approach outperforms transfer learning from ImageNet dataset.Comment: 9 pages, 2 figure

Learning Deep Neural Networks for Enhanced Prostate Histological Image Analysis

In recent years, deep convolutional neural networks (CNNs) have shown promise for improving prostate cancer diagnosis by enabling quantitative histopathology through digital pathology. However, there are a number of factors that limit the widespread adoption and clinical utility of deep learning for digital pathology. One of these limitations is the requirement for large labelled training datasets which are expensive to construct due to limited availability of the requisite expertise. Additionally, digital pathology applications typically require the digitisation of histological slides at high magnifications. This process can be challenging especially when digitising large histological slides such as prostatectomies. This work studies and addresses these issues in two important applications of digital pathology: prostate nuclei detection and cell type classification. We study the performance of CNNs at different magnifications and demonstrate that it is possible to perform nuclei detection in low magnification prostate histopathology using CNNs with minimal loss in accuracy. We then study the training of prostate nuclei detectors in the small data setting and demonstrate that although it is possible to train nuclei detectors with minimal data, the models will be sensitive to hyperparameter choice and therefore may not generalise well. Instead, we show that pre-training the CNNs with colon histology data makes them more robust to hyperparameter choice. We then study the CNN performance for prostate cell type classification using supervised, transfer and semi-supervised learning in the small data setting. Our results show that transfer learning can be detrimental to performance but semi-supervised learning is able to provide significant improvements to the learning curve, allowing the training of neural networks with modest amounts of labelled data. We then propose a novel semi-supervised learning method called Deeply-supervised Exemplar CNNs and demonstrate their ability to improve the cell type classifier learning curves at a much better rate than previous semi-supervised neural network methods

Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions

Breast cancer has reached the highest incidence rate worldwide among all malignancies since 2020. Breast imaging plays a significant role in early diagnosis and intervention to improve the outcome of breast cancer patients. In the past decade, deep learning has shown remarkable progress in breast cancer imaging analysis, holding great promise in interpreting the rich information and complex context of breast imaging modalities. Considering the rapid improvement in the deep learning technology and the increasing severity of breast cancer, it is critical to summarize past progress and identify future challenges to be addressed. In this paper, we provide an extensive survey of deep learning-based breast cancer imaging research, covering studies on mammogram, ultrasound, magnetic resonance imaging, and digital pathology images over the past decade. The major deep learning methods, publicly available datasets, and applications on imaging-based screening, diagnosis, treatment response prediction, and prognosis are described in detail. Drawn from the findings of this survey, we present a comprehensive discussion of the challenges and potential avenues for future research in deep learning-based breast cancer imaging.Comment: Survey, 41 page