Segmentation, Super-resolution and Fusion for Digital Mammogram Classification

Mammography is one of the most common and effective techniques used by radiologists for the early detection of breast cancer. Recently, computer-aided detection/diagnosis (CAD) has become a major research topic in medical imaging and has been widely applied in clinical situations. According to statics, early detection of cancer can reduce the mortality rates by 30% to 70%, therefore detection and diagnosis in the early stage are very important. CAD systems are designed primarily to assist radiologists in detecting and classifying abnormalities in medical scan images, but the main challenges hindering their wider deployment is the difficulty in achieving accuracy rates that help improve radiologists’ performance. The detection and diagnosis of breast cancer face two main issues: the accuracy of the CAD system, and the radiologists’ performance in reading and diagnosing mammograms. This thesis focused on the accuracy of CAD systems. In particular, we investigated two main steps of CAD systems; pre-processing (enhancement and segmentation), feature extraction and classification. Through this investigation, we make five main contributions to the field of automatic mammogram analysis. In automated mammogram analysis, image segmentation techniques are employed in breast boundary or region-of-interest (ROI) extraction. In most Medio-Lateral Oblique (MLO) views of mammograms, the pectoral muscle represents a predominant density region and it is important to detect and segment out this muscle region during pre-processing because it could be bias to the detection of breast cancer. An important reason for the breast border extraction is that it will limit the search-zone for abnormalities in the region of the breast without undue influence from the background of the mammogram. Therefore, we propose a new scheme for breast border extraction, artifact removal and removal of annotations, which are found in the background of mammograms. This was achieved using an local adaptive threshold that creates a binary mask for the images, followed by the use of morphological operations. Furthermore, an adaptive algorithm is proposed to detect and remove the pectoral muscle automatically. Feature extraction is another important step of any image-based pattern classification system. The performance of the corresponding classification depends very much on how well the extracted features represent the object of interest. We investigated a range of different texture feature sets such as Local Binary Pattern Histogram (LBPH), Histogram of Oriented Gradients (HOG) descriptor, and Gray Level Co-occurrence Matrix (GLCM). We propose the use of multi-scale features based on wavelet and local binary patterns for mammogram classification. We extract histograms of LBP codes from the original image as well as the wavelet sub-bands. Extracted features are combined into a single feature set. Experimental results show that our proposed method of combining LBPH features obtained from the original image and with LBPH features obtained from the wavelet domain increase the classification accuracy (sensitivity and specificity) when compared with LBPH extracted from the original image. The feature vector size could be large for some types of feature extraction schemes and they may contain redundant features that could have a negative effect on the performance of classification accuracy. Therefore, feature vector size reduction is needed to achieve higher accuracy as well as efficiency (processing and storage). We reduced the size of the features by applying principle component analysis (PCA) on the feature set and only chose a small number of eigen components to represent the features. Experimental results showed enhancement in the mammogram classification accuracy with a small set of features when compared with using original feature vector. Then we investigated and propose the use of the feature and decision fusion in mammogram classification. In feature-level fusion, two or more extracted feature sets of the same mammogram are concatenated into a single larger fused feature vector to represent the mammogram. Whereas in decision-level fusion, the results of individual classifiers based on distinct features extracted from the same mammogram are combined into a single decision. In this case the final decision is made by majority voting among the results of individual classifiers. Finally, we investigated the use of super resolution as a pre-processing step to enhance the mammograms prior to extracting features. From the preliminary experimental results we conclude that using enhanced mammograms have a positive effect on the performance of the system. Overall, our combination of proposals outperforms several existing schemes published in the literature

Digital Image Processing

This book presents several recent advances that are related or fall under the umbrella of 'digital image processing', with the purpose of providing an insight into the possibilities offered by digital image processing algorithms in various fields. The presented mathematical algorithms are accompanied by graphical representations and illustrative examples for an enhanced readability. The chapters are written in a manner that allows even a reader with basic experience and knowledge in the digital image processing field to properly understand the presented algorithms. Concurrently, the structure of the information in this book is such that fellow scientists will be able to use it to push the development of the presented subjects even further

Innovations in Medical Image Analysis and Explainable AI for Transparent Clinical Decision Support Systems

This thesis explores innovative methods designed to assist clinicians in their everyday practice, with a particular emphasis on Medical Image Analysis and Explainability issues. The main challenge lies in interpreting the knowledge gained from machine learning algorithms, also called black-boxes, to provide transparent clinical decision support systems for real integration into clinical practice. For this reason, all work aims to exploit Explainable AI techniques to study and interpret the trained models. Given the countless open problems for the development of clinical decision support systems, the project includes the analysis of various data and pathologies. The main works are focused on the most threatening disease afflicting the female population: Breast Cancer. The works aim to diagnose and classify breast cancer through medical images by taking advantage of a first-level examination such as Mammography screening, Ultrasound images, and a more advanced examination such as MRI. Papers on Breast Cancer and Microcalcification Classification demonstrated the potential of shallow learning algorithms in terms of explainability and accuracy when intelligible radiomic features are used. Conversely, the union of deep learning and Explainable AI methods showed impressive results for Breast Cancer Detection. The local explanations provided via saliency maps were critical for model introspection, as well as increasing performance. To increase trust in these systems and aspire to their real use, a multi-level explanation was proposed. Three main stakeholders who need transparent models have been identified: developers, physicians, and patients. For this reason, guided by the enormous impact of COVID-19 in the world population, a fully Explainable machine learning model was proposed for COVID-19 Prognosis prediction exploiting the proposed multi-level explanation. It is assumed that such a system primarily requires two components: 1) inherently explainable inputs such as clinical, laboratory, and radiomic features; 2) Explainable methods capable of explaining globally and locally the trained model. The union of these two requirements allows the developer to detect any model bias, the doctor to verify the model findings with clinical evidence, and justify decisions to patients. These results were also confirmed for the study of coronary artery disease. In particular machine learning algorithms are trained using intelligible clinical and radiomic features extracted from pericoronaric adipose tissue to assess the condition of coronary arteries. Eventually, some important national and international collaborations led to the analysis of data for the development of predictive models for some neurological disorders. In particular, the predictivity of handwriting features for the prediction of depressed patients was explored. Using the training of neural networks constrained by first-order logic, it was possible to provide high-performance and explainable models, going beyond the trade-off between explainability and accuracy

Mammography Techniques and Review

Mammography remains at the backbone of medical tools to examine the human breast. The early detection of breast cancer typically uses adjunct tests to mammogram such as ultrasound, positron emission mammography, electrical impedance, Computer-aided detection systems and others. In the present digital era it is even more important to use the best new techniques and systems available to improve the correct diagnosis and to prevent mortality from breast cancer. The first part of this book deals with the electrical impedance mammographic scheme, ultrasound axillary imaging, position emission mammography and digital mammogram enhancement. A detailed consideration of CBR CAD System and the availability of mammographs in Brazil forms the second part of this book. With the up-to-date papers from world experts, this book will be invaluable to anyone who studies the field of mammography

Development of Features and Feature Reduction Techniques for Mammogram Classification

Breast cancer is one of the most widely recognized reasons for increased death rate among women. For reduction of the death rate due to breast cancer, early detection and treatment are of utmost necessity. Recent developments in digital mammography imaging systems have aimed to better diagnosis of abnormalities present in the breast. In the current scenario, mammography is an effectual and reliable method for an accurate detection of breast cancer. Digital mammograms are computerized X-ray images of breasts. Reading of mammograms is a crucial task for radiologists as they suggest patients for biopsy. It has been studied that radiologists report several interpretations for the same mammographic image. Thus, mammogram interpretation is a repetitive task that requires maximum attention for the avoidance of misinterpretation. Therefore, at present, Computer-Aided Diagnosis (CAD) system is exceptionally popular which analyzes the mammograms with the usage of image processing and pattern recognition techniques and classify them into several classes namely, malignant, benign, and normal. The CAD system recognizes the type of tissues automatically by collecting and analyzing significant features from mammographic images. In this thesis, the contributions aim at developing the new and useful features from mammograms for classification of the pattern of tissues. Additionally, some feature reduction techniques have been proposed to select the reduced set of significant features prior to classification. In this context, five different schemes have been proposed for extraction and selection of relevant features for subsequent classification. Using the relevant features, several classifiers are employed for classification of mammograms to derive an overall inference. Each scheme has been validated using two standard databases, namely MIAS and DDSM in isolation. The achieved results are very promising with respect to classification accuracy in comparison to the existing schemes and have been elaborated in each chapter. In Chapter 2, hybrid features are developed using Two-Dimensional Discrete Wavelet Transform (2D-DWT) and Gray-Level Co-occurrence Matrix (GLCM) in succession. Subsequently relevant features are selected using t-test. The resultant feature set is of substantially lower dimension. On application of various classifiers it is observed that Back-Propagation Neural Network (BPNN) gives better classification accuracy as compared to others. In Chapter 3, a Segmentation-based Fractal Texture Analysis (SFTA) is used to extract the texture features from the mammograms. A Fast Correlation-Based Filter (FCBF) method has been used to generate a significant feature subset. Among all classifiers, Support Vector Machine (SVM) results superior classification accuracy. In Chapter 4, Two-Dimensional Discrete Orthonormal S-Transform (2D-DOST) is used to extract the features from mammograms. A feature selection methodology based on null-hypothesis with statistical two-sample t-test method has been suggested to select most significant features. This feature with AdaBoost and Random Forest (AdaBoost-RF) classifier outperforms other classifierswith respect to accuracy. In Chapter 5, features are derived using Two-Dimensional Slantlet Transform (2D-SLT) from mammographic images. The most significant features are selected by utilizing the Bayesian Logistic Regression (BLogR) method. Utilizing these features, LogitBoost and Random Forest (LogitBoost-RF) classifier gives the better classification accuracy among all the classifiers. In Chapter 6, Fast Radial Symmetry Transform (FRST) is applied to mammographic images for derivation of radially symmetric features. A t-distributed Stochastic Neighbor Embedding (t-SNE) method has been utilized to select most relevant features. Using these features, classification experiments have been carried out through all the classifiers. A Logistic Model Tree (LMT) classifier achieves optimal results among all classifiers. An overall comparative analysis has also been made among all our suggested features and feature reduction techniques along with the corresponding classifier where they show superior results

Detection of pathologies in retina digital images an empirical mode decomposition approach

Accurate automatic detection of pathologies in retina digital images offers a promising approach in clinicalapplications. This thesis employs the discrete wavelet transform (DWT) and empirical mode decomposition (EMD) to extract six statistical textural features from retina digital images. The statistical features are the mean, standard deviation, smoothness, third moment, uniformity, and entropy. The purpose is to classify normal and abnormal images. Five different pathologies are considered. They are Artery sheath (Coat’s disease), blot hemorrhage, retinal degeneration (circinates), age-related macular degeneration (drusens), and diabetic retinopathy (microaneurysms and exudates). Four classifiers are employed; including support vector machines (SVM), quadratic discriminant analysis (QDA), k-nearest neighbor algorithm (k-NN), and probabilistic neural networks (PNN). For each experiment, ten random folds are generated to perform cross-validation tests. In order to assess the performance of the classifiers, the average and standard deviation of the correct recognition rate, sensitivity and specificity are computed for each simulation. The experimental results highlight two main conclusions. First, they show the outstanding performance of EMD over DWT with all classifiers. Second, they demonstrate the superiority of the SVM classifier over QDA, k-NN, and PNN. Finally, principal component analysis (PCA) was employed to reduce the number of features in hope to improve the accuracy of classifiers. We find that there is no general and significant improvement of the performance, however. In sum, the EMD-SVM system provides a promising approach for the detection of pathologies in digital retina

An Investigation of Global and Local Radiomic Features for Customized Self-Assessment Mammographic Test Sets for Radiologists in China in Comparison with Those in Australia

Self-assessment test sets have demonstrated being effective tools to improve radiologists’ diagnostic skills through immediate error feedback. Current sets use a one-size-fits-all approach in selecting challenging cases, overlooking cohort-specific weaknesses. This thesis assessed feasibility of using a comprehensive set of handcrafted global radiomic features (Stage 1, Chapter 3) as well as handcrafted (Stage 2, Chapter 4) and deep-learning based (Stage 3, Chapter 5) local radiomic features to identify challenging mammographic cases for Chinese and Australian radiologists. In the first stage, global handcrafted radiomic features and Random Forest models analyzed mammography datasets involving 36 radiologists from China and Australia independently assessing 60 dense mammographic cases. The results were used to build and evaluate models’ performance in case difficulty prediction. The second stage focused on local handcrafted radiomic features, utilizing the same dataset but extracting features from error-related local mammographic areas to analyze features linked to diagnostic errors. The final stage introduced deep learning, specifically Convolutional Neural Network (CNN), using an additional test set and radiologists’ readings to identify features linked to false positive errors. Stage 1 found that global radiomic features effectively detected false positive and false negative errors. Notably, Australian radiologists showed less predictable errors than their Chinese counterparts. Feature normalization did not improve model performance. In Stage 2, the model showed varying success rates in predicting false positives and false negatives among the two cohorts, with specific mammographic regions more prone to errors. In Stage 3, the transferred ResNet-50 architecture performed the best for both cohorts. In conclusion, the thesis affirmed the importance of radiomic features in improving curation of cohort-specific self-assessment mammography test sets

IMAGE UNDERSTANDING OF MOLAR PREGNANCY BASED ON ANOMALIES DETECTION

Cancer occurs when normal cells grow and multiply without normal control. As the cells multiply, they form an area of abnormal cells, known as a tumour. Many tumours exhibit abnormal chromosomal segregation at cell division. These anomalies play an important role in detecting molar pregnancy cancer. Molar pregnancy, also known as hydatidiform mole, can be categorised into partial (PHM) and complete (CHM) mole, persistent gestational trophoblastic and choriocarcinoma. Hydatidiform moles are most commonly found in women under the age of 17 or over the age of 35. Hydatidiform moles can be detected by morphological and histopathological examination. Even experienced pathologists cannot easily classify between complete and partial hydatidiform moles. However, the distinction between complete and partial hydatidiform moles is important in order to recommend the appropriate treatment method. Therefore, research into molar pregnancy image analysis and understanding is critical. The hypothesis of this research project is that an anomaly detection approach to analyse molar pregnancy images can improve image analysis and classification of normal PHM and CHM villi. The primary aim of this research project is to develop a novel method, based on anomaly detection, to identify and classify anomalous villi in molar pregnancy stained images. The novel method is developed to simulate expert pathologists’ approach in diagnosis of anomalous villi. The knowledge and heuristics elicited from two expert pathologists are combined with the morphological domain knowledge of molar pregnancy, to develop a heuristic multi-neural network architecture designed to classify the villi into their appropriated anomalous types. This study confirmed that a single feature cannot give enough discriminative power for villi classification. Whereas expert pathologists consider the size and shape before textural features, this thesis demonstrated that the textural feature has a higher discriminative power than size and shape. The first heuristic-based multi-neural network, which was based on 15 elicited features, achieved an improved average accuracy of 81.2%, compared to the traditional multi-layer perceptron (80.5%); however, the recall of CHM villi class was still low (64.3%). Two further textural features, which were elicited and added to the second heuristic-based multi-neural network, have improved the average accuracy from 81.2% to 86.1% and the recall of CHM villi class from 64.3% to 73.5%. The precision of the multi-neural network II has also increased from 82.7% to 89.5% for normal villi class, from 81.3% to 84.7% for PHM villi class and from 80.8% to 86% for CHM villi class. To support pathologists to visualise the results of the segmentation, a software tool, Hydatidiform Mole Analysis Tool (HYMAT), was developed compiling the morphological and pathological data for each villus analysis

Integrated Graph Theoretic, Radiomics, and Deep Learning Framework for Personalized Clinical Diagnosis, Prognosis, and Treatment Response Assessment of Body Tumors

Purpose: A new paradigm is beginning to emerge in radiology with the advent of increased computational capabilities and algorithms. The future of radiological reading rooms is heading towards a unique collaboration between computer scientists and radiologists. The goal of computational radiology is to probe the underlying tissue using advanced algorithms and imaging parameters and produce a personalized diagnosis that can be correlated to pathology. This thesis presents a complete computational radiology framework (I GRAD) for personalized clinical diagnosis, prognosis and treatment planning using an integration of graph theory, radiomics, and deep learning. Methods: There are three major components of the I GRAD framework–image segmentation, feature extraction, and clinical decision support. Image Segmentation: I developed the multiparametric deep learning (MPDL) tissue signature model for segmentation of normal and abnormal tissue from multiparametric (mp) radiological images. The segmentation MPDL network was constructed from stacked sparse autoencoders (SSAE) with five hidden layers. The MPDL network parameters were optimized using k-fold cross-validation. In addition, the MPDL segmentation network was tested on an independent dataset. Feature Extraction: I developed the radiomic feature mapping (RFM) and contribution scattergram (CSg) methods for characterization of spatial and inter-parametric relationships in multiparametric imaging datasets. The radiomic feature maps were created by filtering radiological images with first and second order statistical texture filters followed by the development of standardized features for radiological correlation to biology and clinical decision support. The contribution scattergram was constructed to visualize and understand the inter-parametric relationships of the breast MRI as a complex network. This multiparametric imaging complex network was modeled using manifold learning and evaluated using graph theoretic analysis. Feature Integration: The different clinical and radiological features extracted from multiparametric radiological images and clinical records were integrated using a hybrid multiview manifold learning technique termed the Informatics Radiomics Integration System (IRIS). IRIS uses hierarchical clustering in combination with manifold learning to visualize the high-dimensional patient space on a two-dimensional heatmap. The heatmap highlights the similarity and dissimilarity between different patients and variables. Results: All the algorithms and techniques presented in this dissertation were developed and validated using breast cancer as a model for diagnosis and prognosis using multiparametric breast magnetic resonance imaging (MRI). The deep learning MPDL method demonstrated excellent dice similarity of 0.87±0.05 and 0.84±0.07 for segmentation of lesions on malignant and benign breast patients, respectively. Furthermore, each of the methods, MPDL, RFM, and CSg demonstrated excellent results for breast cancer diagnosis with area under the receiver (AUC) operating characteristic (ROC) curve of 0.85, 0.91, and 0.87, respectively. Furthermore, IRIS classified patients with low risk of breast cancer recurrence from patients with medium and high risk with an AUC of 0.93 compared to OncotypeDX, a 21 gene assay for breast cancer recurrence. Conclusion: By integrating advanced computer science methods into the radiological setting, the I-GRAD framework presented in this thesis can be used to model radiological imaging data in combination with clinical and histopathological data and produce new tools for personalized diagnosis, prognosis or treatment planning by physicians