Computer-aided detection and diagnosis of breast cancer in 2D and 3D medical imaging through multifractal analysis

This Thesis describes the research work performed in the scope of a doctoral research program and presents its conclusions and contributions. The research activities were carried on in the industry with Siemens S.A. Healthcare Sector, in integration with a research team. Siemens S.A. Healthcare Sector is one of the world biggest suppliers of products, services and complete solutions in the medical sector. The company offers a wide selection of diagnostic and therapeutic equipment and information systems. Siemens products for medical imaging and in vivo diagnostics include: ultrasound, computer tomography, mammography, digital breast tomosynthesis, magnetic resonance, equipment to angiography and coronary angiography, nuclear imaging, and many others. Siemens has a vast experience in Healthcare and at the beginning of this project it was strategically interested in solutions to improve the detection of Breast Cancer, to increase its competitiveness in the sector. The company owns several patents related with self-similarity analysis, which formed the background of this Thesis. Furthermore, Siemens intended to explore commercially the computer- aided automatic detection and diagnosis eld for portfolio integration. Therefore, with the high knowledge acquired by University of Beira Interior in this area together with this Thesis, will allow Siemens to apply the most recent scienti c progress in the detection of the breast cancer, and it is foreseeable that together we can develop a new technology with high potential. The project resulted in the submission of two invention disclosures for evaluation in Siemens A.G., two articles published in peer-reviewed journals indexed in ISI Science Citation Index, two other articles submitted in peer-reviewed journals, and several international conference papers. This work on computer-aided-diagnosis in breast led to innovative software and novel processes of research and development, for which the project received the Siemens Innovation Award in 2012. It was very rewarding to carry on such technological and innovative project in a socially sensitive area as Breast Cancer.No cancro da mama a deteção precoce e o diagnóstico correto são de extrema importância na prescrição terapêutica e caz e e ciente, que potencie o aumento da taxa de sobrevivência à doença. A teoria multifractal foi inicialmente introduzida no contexto da análise de sinal e a sua utilidade foi demonstrada na descrição de comportamentos siológicos de bio-sinais e até na deteção e predição de patologias. Nesta Tese, três métodos multifractais foram estendidos para imagens bi-dimensionais (2D) e comparados na deteção de microcalci cações em mamogramas. Um destes métodos foi também adaptado para a classi cação de massas da mama, em cortes transversais 2D obtidos por ressonância magnética (RM) de mama, em grupos de massas provavelmente benignas e com suspeição de malignidade. Um novo método de análise multifractal usando a lacunaridade tri-dimensional (3D) foi proposto para classi cação de massas da mama em imagens volumétricas 3D de RM de mama. A análise multifractal revelou diferenças na complexidade subjacente às localizações das microcalci cações em relação aos tecidos normais, permitindo uma boa exatidão da sua deteção em mamogramas. Adicionalmente, foram extraídas por análise multifractal características dos tecidos que permitiram identi car os casos tipicamente recomendados para biópsia em imagens 2D de RM de mama. A análise multifractal 3D foi e caz na classi cação de lesões mamárias benignas e malignas em imagens 3D de RM de mama. Este método foi mais exato para esta classi cação do que o método 2D ou o método padrão de análise de contraste cinético tumoral. Em conclusão, a análise multifractal fornece informação útil para deteção auxiliada por computador em mamogra a e diagnóstico auxiliado por computador em imagens 2D e 3D de RM de mama, tendo o potencial de complementar a interpretação dos radiologistas

A Bottom-Up Review of Image Analysis Methods for Suspicious Region Detection in Mammograms.

Breast cancer is one of the most common death causes amongst women all over the world. Early detection of breast cancer plays a critical role in increasing the survival rate. Various imaging modalities, such as mammography, breast MRI, ultrasound and thermography, are used to detect breast cancer. Though there is a considerable success with mammography in biomedical imaging, detecting suspicious areas remains a challenge because, due to the manual examination and variations in shape, size, other mass morphological features, mammography accuracy changes with the density of the breast. Furthermore, going through the analysis of many mammograms per day can be a tedious task for radiologists and practitioners. One of the main objectives of biomedical imaging is to provide radiologists and practitioners with tools to help them identify all suspicious regions in a given image. Computer-aided mass detection in mammograms can serve as a second opinion tool to help radiologists avoid running into oversight errors. The scientific community has made much progress in this topic, and several approaches have been proposed along the way. Following a bottom-up narrative, this paper surveys different scientific methodologies and techniques to detect suspicious regions in mammograms spanning from methods based on low-level image features to the most recent novelties in AI-based approaches. Both theoretical and practical grounds are provided across the paper sections to highlight the pros and cons of different methodologies. The paper's main scope is to let readers embark on a journey through a fully comprehensive description of techniques, strategies and datasets on the topic

Computer Aided Detection of Microcalcifications Utilizing Texture Analysis

A comparative study of texture measures for the classification of breast tissue is presented. The texture features investigated include Angular Second Moments, Power Spectrum Analysis and a novel feature, Laws Energy Ratios. The texture study was accomplished as part of the development of a Model Based Vision (MBV) system for the automatic detection of microcalcifications. An overview of the Microcalcification Detection System is presented, which applies image differencing techniques, feature selection methods, and neural networks for locating microcalcification clusters in mammograms. The Power Spectrum Analysis feature set had the best overall performance with an 83% Probability of Detection and an average False ROl Rate of 2.17 ROIs per image over 53 mammograms. A combination of Laws Energy Ratio and Power Spectrum Analysis features selected using Ruck Saliency metrics achieved an increased Probability of Detection of 85% with an average 4 false ROIs per image

Advanced Computational Methods for Oncological Image Analysis

[Cancer is the second most common cause of death worldwide and encompasses highly variable clinical and biological scenarios. Some of the current clinical challenges are (i) early diagnosis of the disease and (ii) precision medicine, which allows for treatments targeted to specific clinical cases. The ultimate goal is to optimize the clinical workflow by combining accurate diagnosis with the most suitable therapies. Toward this, large-scale machine learning research can define associations among clinical, imaging, and multi-omics studies, making it possible to provide reliable diagnostic and prognostic biomarkers for precision oncology. Such reliable computer-assisted methods (i.e., artificial intelligence) together with clinicians’ unique knowledge can be used to properly handle typical issues in evaluation/quantification procedures (i.e., operator dependence and time-consuming tasks). These technical advances can significantly improve result repeatability in disease diagnosis and guide toward appropriate cancer care. Indeed, the need to apply machine learning and computational intelligence techniques has steadily increased to effectively perform image processing operations—such as segmentation, co-registration, classification, and dimensionality reduction—and multi-omics data integration.

Characterising pattern asymmetry in pigmented skin lesions

Abstract. In clinical diagnosis of pigmented skin lesions asymmetric pigmentation is often indicative of melanoma. This paper describes a method and measures for characterizing lesion symmetry. The estimate of mirror symmetry is computed first for a number of axes at different degrees of rotation with respect to the lesion centre. The statistics of these estimates are the used to assess the overall symmetry. The method is applied to three different lesion representations showing the overall pigmentation, the pigmentation pattern, and the pattern of dermal melanin. The best measure is a 100% sensitive and 96% specific indicator of melanoma on a test set of 33 lesions, with a separate training set consisting of 66 lesions

Developing and Applying CAD-generated Image Markers to Assist Disease Diagnosis and Prognosis Prediction

Developing computer-aided detection and/or diagnosis (CAD) schemes has been an active research topic in medical imaging informatics (MII) with promising results in assisting clinicians in making better diagnostic and/or clinical decisions in the last two decades. To build robust CAD schemes, we need to develop state-of-the-art image processing and machine learning (ML) algorithms to optimize each step in the CAD pipeline, including detection and segmentation of the region of interest, optimal feature generation, followed by integration to ML classifiers. In my dissertation, I conducted multiple studies investigating the feasibility of developing several novel CAD schemes in the field of medicine concerning different purposes. The first study aims to investigate how to optimally develop a CAD scheme of contrast-enhanced digital mammography (CEDM) images to classify breast masses. CEDM includes both low energy (LE) and dual-energy subtracted (DES) images. A CAD scheme was applied to segment mass regions depicting LE and DES images separately. Optimal segmentation results generated from DES images were also mapped to LE images or vice versa. After computing image features, multilayer perceptron-based ML classifiers integrated with a correlation-based feature subset evaluator and leave-one-case-out cross-validation method were built to classify mass regions. The study demonstrated that DES images eliminated the overlapping effect of dense breast tissue, which helps improve mass segmentation accuracy. By mapping mass regions segmented from DES images to LE images, CAD yields significantly improved performance. The second study aims to develop a new quantitative image marker computed from the pre-intervention computed tomography perfusion (CTP) images and evaluate its feasibility to predict clinical outcome among acute ischemic stroke (AIS) patients undergoing endovascular mechanical thrombectomy after diagnosis of large vessel occlusion. A CAD scheme is first developed to pre-process CTP images of different scanning series for each study case, perform image segmentation, quantify contrast-enhanced blood volumes in bilateral cerebral hemispheres, and compute image features related to asymmetrical cerebral blood flow patterns based on the cumulative cerebral blood flow curves of two hemispheres. Next, image markers based on a single optimal feature and ML models fused with multi-features are developed and tested to classify AIS cases into two classes of good and poor prognosis based on the Modified Rankin Scale. The study results show that ML model trained using multiple features yields significantly higher classification performance than the image marker using the best single feature (p<0.01). This study demonstrates the feasibility of developing a new CAD scheme to predict the prognosis of AIS patients in the hyperacute stage, which has the potential to assist clinicians in optimally treating and managing AIS patients. The third study aims to develop and test a new CAD scheme to predict prognosis in aneurysmal subarachnoid hemorrhage (aSAH) patients using brain CT images. Each patient had two sets of CT images acquired at admission and prior to discharge. CAD scheme was applied to segment intracranial brain regions into four subregions, namely, cerebrospinal fluid (CSF), white matter (WM), gray matter (GM), and extraparenchymal blood (EPB), respectively. CAD then computed nine image features related to 5 volumes of the segmented sulci, EPB, CSF, WM, GM, and four volumetrical ratios to sulci. Subsequently, 16 ML models were built using multiple features computed either from CT images acquired at admission or prior to discharge to predict eight prognosis related parameters. The results show that ML models trained using CT images acquired at admission yielded higher accuracy to predict short-term clinical outcomes, while ML models trained using CT images acquired prior to discharge had higher accuracy in predicting long-term clinical outcomes. Thus, this study demonstrated the feasibility of predicting the prognosis of aSAH patients using new ML model-generated quantitative image markers. The fourth study aims to develop and test a new interactive computer-aided detection (ICAD) tool to quantitatively assess hemorrhage volumes. After loading each case, the ICAD tool first segments intracranial brain volume, performs CT labeling of each voxel. Next, contour-guided image-thresholding techniques based on CT Hounsfield Unit are used to estimate and segment hemorrhage-associated voxels (ICH). Next, two experienced neurology residents examine and correct the markings of ICH categorized into either intraparenchymal hemorrhage (IPH) or intraventricular hemorrhage (IVH) to obtain the true markings. Additionally, volumes and maximum two-dimensional diameter of each sub-type of hemorrhage are also computed for understanding ICH prognosis. The performance to segment hemorrhage regions between semi-automated ICAD and the verified neurology residents’ true markings is evaluated using dice similarity coefficient (DSC). The data analysis results in the study demonstrate that the new ICAD tool enables to segment and quantify ICH and other hemorrhage volumes with higher DSC. Finally, the fifth study aims to bridge the gap between traditional radiomics and deep learning systems by comparing and assessing these two technologies in classifying breast lesions. First, one CAD scheme is applied to segment lesions and compute radiomics features. In contrast, another scheme applies a pre-trained residual net architecture (ResNet50) as a transfer learning model to extract automated features. Next, the principal component algorithm processes both initially computed radiomics and automated features to create optimal feature vectors. Then, several support vector machine (SVM) classifiers are built using the optimized radiomics or automated features. This study indicates that (1) CAD built using only deep transfer learning yields higher classification performance than the traditional radiomic-based model, (2) SVM trained using the fused radiomics and automated features does not yield significantly higher AUC, and (3) radiomics and automated features contain highly correlated information in lesion classification. In summary, in all these studies, I developed and investigated several key concepts of CAD pipeline, including (i) pre-processing algorithms, (ii) automatic detection and segmentation schemes, (iii) feature extraction and optimization methods, and (iv) ML and data analysis models. All developed CAD models are embedded with interactive and visually aided graphical user interfaces (GUIs) to provide user functionality. These techniques present innovative approaches for building quantitative image markers to build optimal ML models. The study results indicate the underlying CAD scheme's potential application to assist radiologists in clinical settings for their assessments in diagnosing disease and improving their overall performance

Integrated Graph Theoretic, Radiomics, and Deep Learning Framework for Personalized Clinical Diagnosis, Prognosis, and Treatment Response Assessment of Body Tumors

Purpose: A new paradigm is beginning to emerge in radiology with the advent of increased computational capabilities and algorithms. The future of radiological reading rooms is heading towards a unique collaboration between computer scientists and radiologists. The goal of computational radiology is to probe the underlying tissue using advanced algorithms and imaging parameters and produce a personalized diagnosis that can be correlated to pathology. This thesis presents a complete computational radiology framework (I GRAD) for personalized clinical diagnosis, prognosis and treatment planning using an integration of graph theory, radiomics, and deep learning. Methods: There are three major components of the I GRAD framework–image segmentation, feature extraction, and clinical decision support. Image Segmentation: I developed the multiparametric deep learning (MPDL) tissue signature model for segmentation of normal and abnormal tissue from multiparametric (mp) radiological images. The segmentation MPDL network was constructed from stacked sparse autoencoders (SSAE) with five hidden layers. The MPDL network parameters were optimized using k-fold cross-validation. In addition, the MPDL segmentation network was tested on an independent dataset. Feature Extraction: I developed the radiomic feature mapping (RFM) and contribution scattergram (CSg) methods for characterization of spatial and inter-parametric relationships in multiparametric imaging datasets. The radiomic feature maps were created by filtering radiological images with first and second order statistical texture filters followed by the development of standardized features for radiological correlation to biology and clinical decision support. The contribution scattergram was constructed to visualize and understand the inter-parametric relationships of the breast MRI as a complex network. This multiparametric imaging complex network was modeled using manifold learning and evaluated using graph theoretic analysis. Feature Integration: The different clinical and radiological features extracted from multiparametric radiological images and clinical records were integrated using a hybrid multiview manifold learning technique termed the Informatics Radiomics Integration System (IRIS). IRIS uses hierarchical clustering in combination with manifold learning to visualize the high-dimensional patient space on a two-dimensional heatmap. The heatmap highlights the similarity and dissimilarity between different patients and variables. Results: All the algorithms and techniques presented in this dissertation were developed and validated using breast cancer as a model for diagnosis and prognosis using multiparametric breast magnetic resonance imaging (MRI). The deep learning MPDL method demonstrated excellent dice similarity of 0.87±0.05 and 0.84±0.07 for segmentation of lesions on malignant and benign breast patients, respectively. Furthermore, each of the methods, MPDL, RFM, and CSg demonstrated excellent results for breast cancer diagnosis with area under the receiver (AUC) operating characteristic (ROC) curve of 0.85, 0.91, and 0.87, respectively. Furthermore, IRIS classified patients with low risk of breast cancer recurrence from patients with medium and high risk with an AUC of 0.93 compared to OncotypeDX, a 21 gene assay for breast cancer recurrence. Conclusion: By integrating advanced computer science methods into the radiological setting, the I-GRAD framework presented in this thesis can be used to model radiological imaging data in combination with clinical and histopathological data and produce new tools for personalized diagnosis, prognosis or treatment planning by physicians