A study of generative adversarial networks to improve classification of microscopic foraminifera

Foraminifera are single-celled organisms with shells that live in the marine environment and can be found abundantly as fossils in e.g. sediment cores. The assemblages of different species and their numbers serves as an important source of data for marine, geological, climate and environmental research. Steps towards automatic classification of foraminifera using deep learning (DL) models have been made (Johansen and Sørensen, 2020), and this thesis sets out to improve the accuracy of their proposed model. The recent advances of DL models such as generative adversarial networks (GANs) (Goodfellow et al., 2014), and their ability to model high-dimensional distributions such as real-world images, are used to achieve this objective. GANs are studied and explored from a theoretical and empirical standpoint to uncover how they can be used to generate images of foraminifera. A multi-scale gradient GAN is implemented, tested and trained to learn the distributions of four high-level classes of a recent foraminifera dataset (Johansen and Sørensen, 2020), both conditionally and unconditionally. The conditional images are assessed by an expert and a deep learning classification model and is found to contain mostly valuable characteristics, although some artificial artifacts are introduced. The unconditional images measured a Fréchet Inception distance of 47.1. From the conditionally learned distributions a total of 10 000 images are sampled from the four distributions. These images are used to augment the original foraminifera training set in an attempt to improve the classification accuracy of (Johansen and Sørensen, 2020). Due to limitations of computational resources, the experiments were carried out with images of resolution 128 × 128. The synthetic image augmentation lead to an improvement in mean accuracy from 97.3 ± 0.4 % to 97.4 ± 0.7 % and an improvement in best achieved accuracy from 97.7 % to 98.5 %

View it like a radiologist: Shifted windows for deep learning augmentation of CT images

Deep learning has the potential to revolutionize medical practice by automating and performing important tasks like detecting and delineating the size and locations of cancers in medical images. However, most deep learning models rely on augmentation techniques that treat medical images as natural images. For contrast-enhanced Computed Tomography (CT) images in particular, the signals producing the voxel intensities have physical meaning, which is lost during preprocessing and augmentation when treating such images as natural images. To address this, we propose a novel preprocessing and intensity augmentation scheme inspired by how radiologists leverage multiple viewing windows when evaluating CT images. Our proposed method, window shifting, randomly places the viewing windows around the region of interest during training. This approach improves liver lesion segmentation performance and robustness on images with poorly timed contrast agent. Our method outperforms classical intensity augmentations as well as the intensity augmentation pipeline of the popular nn-UNet on multiple datasets

A clinically motivated self-supervised approach for content-based image retrieval of CT liver images

Deep learning-based approaches for content-based image retrieval (CBIR) of computed tomography (CT) liver images is an active field of research, but suffer from some critical limitations. First, they are heavily reliant on labeled data, which can be challenging and costly to acquire. Second, they lack transparency and explainability, which limits the trustworthiness of deep CBIR systems. We address these limitations by: (1) Proposing a self-supervised learning framework that incorporates domain-knowledge into the training procedure, and, (2) by providing the first representation learning explainability analysis in the context of CBIR of CT liver images. Results demonstrate improved performance compared to the standard self-supervised approach across several metrics, as well as improved generalization across datasets. Further, we conduct the first representation learning explainability analysis in the context of CBIR, which reveals new insights into the feature extraction process. Lastly, we perform a case study with cross-examination CBIR that demonstrates the usability of our proposed framework. We believe that our proposed framework could play a vital role in creating trustworthy deep CBIR systems that can successfully take advantage of unlabeled data

A study of generative adversarial networks to improve classification of microscopic foraminifera

Foraminifera are single-celled organisms with shells that live in the marine environment and can be found abundantly as fossils in e.g. sediment cores. The assemblages of different species and their numbers serves as an important source of data for marine, geological, climate and environmental research. Steps towards automatic classification of foraminifera using deep learning (DL) models have been made (Johansen and Sørensen, 2020), and this thesis sets out to improve the accuracy of their proposed model. The recent advances of DL models such as generative adversarial networks (GANs) (Goodfellow et al., 2014), and their ability to model high-dimensional distributions such as real-world images, are used to achieve this objective. GANs are studied and explored from a theoretical and empirical standpoint to uncover how they can be used to generate images of foraminifera. A multi-scale gradient GAN is implemented, tested and trained to learn the distributions of four high-level classes of a recent foraminifera dataset (Johansen and Sørensen, 2020), both conditionally and unconditionally. The conditional images are assessed by an expert and a deep learning classification model and is found to contain mostly valuable characteristics, although some artificial artifacts are introduced. The unconditional images measured a Fréchet Inception distance of 47.1. From the conditionally learned distributions a total of 10 000 images are sampled from the four distributions. These images are used to augment the original foraminifera training set in an attempt to improve the classification accuracy of (Johansen and Sørensen, 2020). Due to limitations of computational resources, the experiments were carried out with images of resolution 128 × 128. The synthetic image augmentation lead to an improvement in mean accuracy from 97.3 ± 0.4 % to 97.4 ± 0.7 % and an improvement in best achieved accuracy from 97.7 % to 98.5 %