Computer-aided detection and diagnosis of breast cancer in 2D and 3D medical imaging through multifractal analysis

This Thesis describes the research work performed in the scope of a doctoral research program and presents its conclusions and contributions. The research activities were carried on in the industry with Siemens S.A. Healthcare Sector, in integration with a research team. Siemens S.A. Healthcare Sector is one of the world biggest suppliers of products, services and complete solutions in the medical sector. The company offers a wide selection of diagnostic and therapeutic equipment and information systems. Siemens products for medical imaging and in vivo diagnostics include: ultrasound, computer tomography, mammography, digital breast tomosynthesis, magnetic resonance, equipment to angiography and coronary angiography, nuclear imaging, and many others. Siemens has a vast experience in Healthcare and at the beginning of this project it was strategically interested in solutions to improve the detection of Breast Cancer, to increase its competitiveness in the sector. The company owns several patents related with self-similarity analysis, which formed the background of this Thesis. Furthermore, Siemens intended to explore commercially the computer- aided automatic detection and diagnosis eld for portfolio integration. Therefore, with the high knowledge acquired by University of Beira Interior in this area together with this Thesis, will allow Siemens to apply the most recent scienti c progress in the detection of the breast cancer, and it is foreseeable that together we can develop a new technology with high potential. The project resulted in the submission of two invention disclosures for evaluation in Siemens A.G., two articles published in peer-reviewed journals indexed in ISI Science Citation Index, two other articles submitted in peer-reviewed journals, and several international conference papers. This work on computer-aided-diagnosis in breast led to innovative software and novel processes of research and development, for which the project received the Siemens Innovation Award in 2012. It was very rewarding to carry on such technological and innovative project in a socially sensitive area as Breast Cancer.No cancro da mama a deteção precoce e o diagnóstico correto são de extrema importância na prescrição terapêutica e caz e e ciente, que potencie o aumento da taxa de sobrevivência à doença. A teoria multifractal foi inicialmente introduzida no contexto da análise de sinal e a sua utilidade foi demonstrada na descrição de comportamentos siológicos de bio-sinais e até na deteção e predição de patologias. Nesta Tese, três métodos multifractais foram estendidos para imagens bi-dimensionais (2D) e comparados na deteção de microcalci cações em mamogramas. Um destes métodos foi também adaptado para a classi cação de massas da mama, em cortes transversais 2D obtidos por ressonância magnética (RM) de mama, em grupos de massas provavelmente benignas e com suspeição de malignidade. Um novo método de análise multifractal usando a lacunaridade tri-dimensional (3D) foi proposto para classi cação de massas da mama em imagens volumétricas 3D de RM de mama. A análise multifractal revelou diferenças na complexidade subjacente às localizações das microcalci cações em relação aos tecidos normais, permitindo uma boa exatidão da sua deteção em mamogramas. Adicionalmente, foram extraídas por análise multifractal características dos tecidos que permitiram identi car os casos tipicamente recomendados para biópsia em imagens 2D de RM de mama. A análise multifractal 3D foi e caz na classi cação de lesões mamárias benignas e malignas em imagens 3D de RM de mama. Este método foi mais exato para esta classi cação do que o método 2D ou o método padrão de análise de contraste cinético tumoral. Em conclusão, a análise multifractal fornece informação útil para deteção auxiliada por computador em mamogra a e diagnóstico auxiliado por computador em imagens 2D e 3D de RM de mama, tendo o potencial de complementar a interpretação dos radiologistas

Pattern classification approaches for breast cancer identification via MRI: state‐of‐the‐art and vision for the future

Mining algorithms for Dynamic Contrast Enhanced Magnetic Resonance Imaging (DCEMRI) of breast tissue are discussed. The algorithms are based on recent advances in multidimensional signal processing and aim to advance current state‐of‐the‐art computer‐aided detection and analysis of breast tumours when these are observed at various states of development. The topics discussed include image feature extraction, information fusion using radiomics, multi‐parametric computer‐aided classification and diagnosis using information fusion of tensorial datasets as well as Clifford algebra based classification approaches and convolutional neural network deep learning methodologies. The discussion also extends to semi‐supervised deep learning and self‐supervised strategies as well as generative adversarial networks and algorithms using generated confrontational learning approaches. In order to address the problem of weakly labelled tumour images, generative adversarial deep learning strategies are considered for the classification of different tumour types. The proposed data fusion approaches provide a novel Artificial Intelligence (AI) based framework for more robust image registration that can potentially advance the early identification of heterogeneous tumour types, even when the associated imaged organs are registered as separate entities embedded in more complex geometric spaces. Finally, the general structure of a high‐dimensional medical imaging analysis platform that is based on multi‐task detection and learning is proposed as a way forward. The proposed algorithm makes use of novel loss functions that form the building blocks for a generated confrontation learning methodology that can be used for tensorial DCE‐MRI. Since some of the approaches discussed are also based on time‐lapse imaging, conclusions on the rate of proliferation of the disease can be made possible. The proposed framework can potentially reduce the costs associated with the interpretation of medical images by providing automated, faster and more consistent diagnosis

Tensor based multichannel reconstruction for breast tumours identification from DCE-MRIs

A new methodology based on tensor algebra that uses a higher order singular value decomposition to perform three-dimensional voxel reconstruction from a series of temporal images obtained using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is proposed. Principal component analysis (PCA) is used to robustly extract the spatial and temporal image features and simultaneously de-noise the datasets. Tumour segmentation on enhanced scaled (ES) images performed using a fuzzy C-means (FCM) cluster algorithm is compared with that achieved using the proposed tensorial framework. The proposed algorithm explores the correlations between spatial and temporal features in the tumours. The multi-channel reconstruction enables improved breast tumour identification through enhanced de-noising and improved intensity consistency. The reconstructed tumours have clear and continuous boundaries; furthermore the reconstruction shows better voxel clustering in tumour regions of interest. A more homogenous intensity distribution is also observed, enabling improved image contrast between tumours and background, especially in places where fatty tissue is imaged. The fidelity of reconstruction is further evaluated on the basis of five new qualitative metrics. Results confirm the superiority of the tensorial approach. The proposed reconstruction metrics should also find future applications in the assessment of other reconstruction algorithms

Advanced perfusion quantification methods for dynamic PET and MRI data modelling

The functionality of tissues is guaranteed by the capillaries, which supply the microvascular network providing a considerable surface area for exchanges between blood and tissues. Microcirculation is affected by any pathological condition and any change in the blood supply can be used as a biomarker for the diagnosis of lesions and the optimization of the treatment. Nowadays, a number of techniques for the study of perfusion in vivo and in vitro are available. Among the several imaging modalities developed for the study of microcirculation, the analysis of the tissue kinetics of intravenously injected contrast agents or tracers is the most widely used technique. Tissue kinetics can be studied using different modalities: the positive enhancement of the signal in the computed tomography and in the ultrasound dynamic contrast enhancement imaging; T1-weighted MRI or the negative enhancement of T2* weighted MRI signal for the dynamic susceptibility contrast imaging or, finally, the uptake of radiolabelled tracers in dynamic PET imaging. Here we will focus on the perfusion quantification of dynamic PET and MRI data. The kinetics of the contrast agent (or the tracer) can be analysed visually, to define qualitative criteria but, traditionally, quantitative physiological parameters are extracted with the implementation of mathematical models. Serial measurements of the concentration of the tracer (or of the contrast agent) in the tissue of interest, together with the knowledge of an arterial input function, are necessary for the calculation of blood flow or perfusion rates from the wash-in and/or wash-out kinetic rate constants. The results depend on the acquisition conditions (type of imaging device, imaging mode, frequency and total duration of the acquisition), the type of contrast agent or tracer used, the data pre-processing (motion correction, attenuation correction, correction of the signal into concentration) and the data analysis method. As for the MRI, dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is a non-invasive imaging technique that can be used to measure properties of tissue microvasculature. It is sensitive to differences in blood volume and vascular permeability that can be associated with tumour angiogenesis. DCE-MRI has been investigated for a range of clinical oncologic applications (breast, prostate, cervix, liver, lung, and rectum) including cancer detection, diagnosis, staging, and assessment of treatment response. Tumour microvascular measurements by DCE-MRI have been found to correlate with prognostic factors (such as tumour grade, microvessel density, and vascular endothelial growth factor expression) and with recurrence and survival outcomes. Furthermore, DCE-MRI changes measured during treatment have been shown to correlate with outcome, suggesting a role as a predictive marker. The accuracy of DCE-MRI relies on the ability to model the pharmacokinetics of an injected contrast agent using the signal intensity changes on sequential magnetic resonance images. DCE-MRI data are usually quantified with the application of the pharmacokinetic two-compartment Tofts model (also known as the standard model), which represents the system with the plasma and tissue (extravascular extracellular space) compartments and with the contrast reagent exchange rates between them. This model assumes a negligible contribution from the vascular space and considers the system in, what-is-known as, the fast exchange limit, assuming infinitely fast transcytolemmal water exchange kinetics. In general, the number, as well as any assumption about the compartments, depends on the properties of the contrast agent used (mainly gadolinium) together with the tissue physiology or pathology studied. For this reason, the choice of the model is crucial in the analysis of DCE-MRI data. The value of PET in clinical oncology has been demonstrated with studies in a variety of cancers including colorectal carcinomas, lung tumours, head and neck tumours, primary and metastatic brain tumours, breast carcinoma, lymphoma, melanoma, bone cancers, and other soft-tissue cancers. PET studies of tumours can be performed for several reasons including the quantification of tumour perfusion, the evaluation of tumour metabolism, the tracing of radiolabelled cytostatic agents. In particular, the kinetic analysis of PET imaging has showed, in the past few years, an increasing value in tumour diagnosis, as well as in tumour therapy, through providing additional indicative parameters. Many authors have showed the benefit of kinetic analysis of anticancer drugs after labelling with radionuclide in measuring the specific therapeutic effect bringing to light the feasibility of applying the kinetic analysis to the dynamic acquisition. Quantification methods can involve visual analysis together with compartmental modelling and can be applied to a wide range of different tracers. The increased glycolysis in the most malignancies makes 18F-FDG-PET the most common diagnostic method used in tumour imaging. But, PET metabolic alteration in the target tissue can depend by many other factors. For example, most types of cancer are characterized by increased choline transport and by the overexpression of choline kinase in highly proliferating cells in response to enhanced demand of phosphatidylcholine (prostate, breast, lung, ovarian and colon cancers). This effect can be diagnosed with choline-based tracers as the 18Ffluoromethylcholine (18F-FCH), or the even more stable 18F-D4-Choline. Cellular proliferation is also imaged with 18F-fluorothymidine (FLT), which is trapped within the cytosol after being mono phosphorylated by thymidine kinase-1 (TK1), a principal enzyme in the salvage pathway of DNA synthesis. 18F-FLT has been found to be useful for noninvasive assessment of the proliferation rate of several types of cancer and showed high reproducibility and accuracy in breast and lung cancer tumours. The aim of this thesis is the perfusion quantification of dynamic PET and MRI data of patients with lung, brain, liver, prostate and breast lesions with the application of advanced models. This study covers a wide range of imaging methods and applications, presenting a novel combination of MRI-based perfusion measures with PET kinetic modelling parameters in oncology. It assesses the applicability and stability of perfusion quantification methods, which are not currently used in the routine clinical practice. The main achievements of this work include: 1) the assessment of the stability of perfusion quantification of D4-Choline and 18F-FLT dynamic PET data in lung and liver lesions, respectively (first applications in the literature); 2) the development of a model selection in the analysis of DCE-MRI data of primary brain tumours (first application of the extended shutter speed model); 3) the multiparametric analysis of PET and MRI derived perfusion measurements of primary brain tumour and breast cancer together with the integration of immuohistochemical markers in the prediction of breast cancer subtype (analysis of data acquired on the hybrid PET/MRI scanner). The thesis is structured as follows: - Chapter 1 is an introductive chapter on cancer biology. Basic concepts, including the causes of cancer, cancer hallmarks, available cancer treatments, are described in this first chapter. Furthermore, there are basic concepts of brain, breast, prostate and lung cancers (which are the lesions that have been analysed in this work). - Chapter 2 is about Positron Emission Tomography. After a brief introduction on the basics of PET imaging, together with data acquisition and reconstruction methods, the chapter focuses on PET in the clinical settings. In particular, it shows the quantification techniques of static and dynamic PET data and my results of the application of graphical methods, spectral analysis and compartmental models on dynamic 18F-FDG, 18F-FLT and 18F-D4- Choline PET data of patients with breast, lung cancer and hepatocellular carcinoma. - Chapter 3 is about Magnetic Resonance Imaging. After a brief introduction on the basics of MRI, the chapter focuses on the quantification of perfusion weighted MRI data. In particular, it shows the pharmacokinetic models for the quantification of dynamic contrast enhanced MRI data and my results of the application of the Tofts, the extended Tofts, the shutter speed and the extended shutter speed models on a dataset of patients with brain glioma. - Chapter 4 introduces the multiparametric imaging techniques, in particular the combined PET/CT and the hybrid PET/MRI systems. The last part of the chapter shows the applications of perfusion quantification techniques on a multiparametric study of breast tumour patients, who simultaneously underwent DCE-MRI and 18F-FDG PET on a hybrid PET/MRI scanner. Then the results of a predictive study on the same dataset of breast tumour patients integrated with immunohistochemical markers. Furthermore, the results of a multiparametric study on DCE-MRI and 18F-FCM brain data acquired both on a PET/CT scanner and on an MR scanner, separately. Finally, it will show the application of kinetic analysis in a radiomic study of patients with prostate cancer

Analysis of contrast-enhanced medical images.

Early detection of human organ diseases is of great importance for the accurate diagnosis and institution of appropriate therapies. This can potentially prevent progression to end-stage disease by detecting precursors that evaluate organ functionality. In addition, it also assists the clinicians for therapy evaluation, tracking diseases progression, and surgery operations. Advances in functional and contrast-enhanced (CE) medical images enabled accurate noninvasive evaluation of organ functionality due to their ability to provide superior anatomical and functional information about the tissue-of-interest. The main objective of this dissertation is to develop a computer-aided diagnostic (CAD) system for analyzing complex data from CE magnetic resonance imaging (MRI). The developed CAD system has been tested in three case studies: (i) early detection of acute renal transplant rejection, (ii) evaluation of myocardial perfusion in patients with ischemic heart disease after heart attack; and (iii), early detection of prostate cancer. However, developing a noninvasive CAD system for the analysis of CE medical images is subject to multiple challenges, including, but are not limited to, image noise and inhomogeneity, nonlinear signal intensity changes of the images over the time course of data acquisition, appearances and shape changes (deformations) of the organ-of-interest during data acquisition, determination of the best features (indexes) that describe the perfusion of a contrast agent (CA) into the tissue. To address these challenges, this dissertation focuses on building new mathematical models and learning techniques that facilitate accurate analysis of CAs perfusion in living organs and include: (i) accurate mathematical models for the segmentation of the object-of-interest, which integrate object shape and appearance features in terms of pixel/voxel-wise image intensities and their spatial interactions; (ii) motion correction techniques that combine both global and local models, which exploit geometric features, rather than image intensities to avoid problems associated with nonlinear intensity variations of the CE images; (iii) fusion of multiple features using the genetic algorithm. The proposed techniques have been integrated into CAD systems that have been tested in, but not limited to, three clinical studies. First, a noninvasive CAD system is proposed for the early and accurate diagnosis of acute renal transplant rejection using dynamic contrast-enhanced MRI (DCE-MRI). Acute rejection–the immunological response of the human immune system to a foreign kidney–is the most sever cause of renal dysfunction among other diagnostic possibilities, including acute tubular necrosis and immune drug toxicity. In the U.S., approximately 17,736 renal transplants are performed annually, and given the limited number of donors, transplanted kidney salvage is an important medical concern. Thus far, biopsy remains the gold standard for the assessment of renal transplant dysfunction, but only as the last resort because of its invasive nature, high cost, and potential morbidity rates. The diagnostic results of the proposed CAD system, based on the analysis of 50 independent in-vivo cases were 96% with a 95% confidence interval. These results clearly demonstrate the promise of the proposed image-based diagnostic CAD system as a supplement to the current technologies, such as nuclear imaging and ultrasonography, to determine the type of kidney dysfunction. Second, a comprehensive CAD system is developed for the characterization of myocardial perfusion and clinical status in heart failure and novel myoregeneration therapy using cardiac first-pass MRI (FP-MRI). Heart failure is considered the most important cause of morbidity and mortality in cardiovascular disease, which affects approximately 6 million U.S. patients annually. Ischemic heart disease is considered the most common underlying cause of heart failure. Therefore, the detection of the heart failure in its earliest forms is essential to prevent its relentless progression to premature death. While current medical studies focus on detecting pathological tissue and assessing contractile function of the diseased heart, this dissertation address the key issue of the effects of the myoregeneration therapy on the associated blood nutrient supply. Quantitative and qualitative assessment in a cohort of 24 perfusion data sets demonstrated the ability of the proposed framework to reveal regional perfusion improvements with therapy, and transmural perfusion differences across the myocardial wall; thus, it can aid in follow-up on treatment for patients undergoing the myoregeneration therapy. Finally, an image-based CAD system for early detection of prostate cancer using DCE-MRI is introduced. Prostate cancer is the most frequently diagnosed malignancy among men and remains the second leading cause of cancer-related death in the USA with more than 238,000 new cases and a mortality rate of about 30,000 in 2013. Therefore, early diagnosis of prostate cancer can improve the effectiveness of treatment and increase the patient’s chance of survival. Currently, needle biopsy is the gold standard for the diagnosis of prostate cancer. However, it is an invasive procedure with high costs and potential morbidity rates. Additionally, it has a higher possibility of producing false positive diagnosis due to relatively small needle biopsy samples. Application of the proposed CAD yield promising results in a cohort of 30 patients that would, in the near future, represent a supplement of the current technologies to determine prostate cancer type. The developed techniques have been compared to the state-of-the-art methods and demonstrated higher accuracy as shown in this dissertation. The proposed models (higher-order spatial interaction models, shape models, motion correction models, and perfusion analysis models) can be used in many of today’s CAD applications for early detection of a variety of diseases and medical conditions, and are expected to notably amplify the accuracy of CAD decisions based on the automated analysis of CE images

PET-MR Imaging of Hypoxia and Vascularity in Breast Cancer

Breast cancer is the most common cancer in the UK and in women globally. Imaging methods like mammography, ultrasound (US) and magnetic resonance imaging (MRI) play an important role in the diagnosis and management of breast cancer; they are generally utilised to provide anatomical or structural description of tumours in the clinical setting. It is widely accepted that the tumour microenvironment influences the phenotype, progression and treatment of breast cancer. This gave the impetus to move beyond tumour visualization in images to radiomics in order to provide additional disease characterisation and early biomarkers of tumour response. Due to their ability to assess physiological processes in vivo, positron emission tomography (PET) and MRI can provide non-invasive characterisation of the tumour microenvironment, including perfusion, vascular permeability, cellularity and hypoxia, which is associated with poor clinical outcome and metastasis. Clinical imaging studies in breast tumours have hitherto assessed tumour physiological parameters separately, with only few directly comparing data from these modalities. To this end, hybrid PET-MRI represents an attractive option as it can allow examination of functional processes and features of tumours simultaneously, while also conferring methodological advantages to the way imaging information is combined. The main aim of this thesis is to provide a better understanding of breast cancer pathophysiology using simultaneous PET and multi-parametric MRI. In particular, this work aims to explore relationships between imaging biomarkers of tumour vascularity measured by dynamic contrast-enhanced (DCE) MRI, cellularity using diffusion-weighted imaging (DWI) and hypoxic status using 18F-fluoromisonidazole (18F-FMISO) PET. Correlations between functional PET-MRI parameters and immunohistochemical (IHC) biomarkers of hypoxia and vascularity as well as MRI morphological tumour descriptors are also presented. The thesis concludes with an investigation of the utility of MRI markers of perfusion and surrogate markers of hypoxia to quantitatively monitor and predict pathological response in patients undergoing neoadjuvant chemotherapy (NACT) and provides projections for future work