Developing Novel Computer Aided Diagnosis Schemes for Improved Classification of Mammography Detected Masses

Mammography imaging is a population-based breast cancer screening tool that has greatly aided in the decrease in breast cancer mortality over time. Although mammography is the most frequently employed breast imaging modality, its performance is often unsatisfactory with low sensitivity and high false positive rates. This is due to the fact that reading and interpreting mammography images remains difficult due to the heterogeneity of breast tumors and dense overlapping fibroglandular tissue. To help overcome these clinical challenges, researchers have made great efforts to develop computer-aided detection and/or diagnosis (CAD) schemes to provide radiologists with decision-making support tools. In this dissertation, I investigate several novel methods for improving the performance of a CAD system in distinguishing between malignant and benign masses. The first study, we test the hypothesis that handcrafted radiomics features and deep learning features contain complementary information, therefore the fusion of these two types of features will increase the feature representation of each mass and improve the performance of CAD system in distinguishing malignant and benign masses. Regions of interest (ROI) surrounding suspicious masses are extracted and two types of features are computed. The first set consists of 40 radiomic features and the second set includes deep learning (DL) features computed from a pretrained VGG16 network. DL features are extracted from two pseudo color image sets, producing a total of three feature vectors after feature extraction, namely: handcrafted, DL-stacked, DL-pseudo. Linear support vector machines (SVM) are trained using each feature set alone and in combinations. Results show that the fusion CAD system significantly outperforms the systems using either feature type alone (AUC=0.756±0.042 p<0.05). This study demonstrates that both handcrafted and DL futures contain useful complementary information and that fusion of these two types of features increases the CAD classification performance. In the second study, we expand upon our first study and develop a novel CAD framework that fuses information extracted from ipsilateral views of bilateral mammograms using both DL and radiomics feature extraction methods. Each case in this study is represented by four images which includes the craniocaudal (CC) and mediolateral oblique (MLO) view of left and right breast. First, we extract matching ROIs from each of the four views using an ipsilateral matching and bilateral registration scheme to ensure masses are appropriately matched. Next, the handcrafted radiomics features and VGG16 model-generated features are extracted from each ROI resulting in eight feature vectors. Then, after reducing feature dimensionality and quantifying the bilateral asymmetry, we test four fusion methods. Results show that multi-view CAD systems significantly outperform single-view systems (AUC = 0.876±0.031 vs AUC = 0.817±0.026 for CC view and 0.792±0.026 for MLO view, p<0.001). The study demonstrates that the shift from single-view CAD to four-view CAD and the inclusion of both deep transfer learning and radiomics features increases the feature representation of the mass thus improves CAD performance in distinguishing between malignant and benign breast lesions. In the third study, we build upon the first and second studies and investigate the effects of pseudo color image generation in classifying suspicious mammography detected breast lesions as malignant or benign using deep transfer learning in a multi-view CAD scheme. Seven pseudo color image sets are created through a combination of the original grayscale image, a histogram equalized image, a bilaterally filtered image, and a segmented mass image. Using the multi-view CAD framework developed in the previous study, we observe that the two pseudo-color sets created using a segmented mass in one of the three image channels performed significantly better than all other pseudo-color sets (AUC=0.882, p<0.05 for all comparisons and AUC=0.889, p<0.05 for all comparisons). The results of this study support our hypothesis that pseudo color images generated with a segmented mass optimize the mammogram image feature representation by providing increased complementary information to the CADx scheme which results in an increase in the performance in classifying suspicious mammography detected breast lesions as malignant or benign. In summary, each of the studies presented in this dissertation aim to increase the accuracy of a CAD system in classifying suspicious mammography detected masses. Each of these studies takes a novel approach to increase the feature representation of the mass that needs to be classified. The results of each study demonstrate the potential utility of these CAD schemes as an aid to radiologists in the clinical workflow

Investigation of physical processes in digital x-ray tomosynthesis imaging of the breast

Early detection is one of the most important factors in the survival of patients diagnosed with breast cancer. For this reason the development of improved screening mammography methods is one of primary importance. One problem that is present in standard planar mammography, which is not solved with the introduction of digital mammography, is the possible masking of lesions by normal breast tissue because of the inherent collapse of three-dimensional anatomy into a two-dimensional image. Digital tomosynthesis imaging has the potential to avoid this effect by incorporating into the acquired image information on the vertical position of the features present in the breast. Previous studies have shown that at an approximately equivalent dose, the contrast-detail trends of several tomosynthesis methods are better than those of planar mammography. By optimizing the image acquisition parameters and the tomosynthesis reconstruction algorithm, it is believed that a tomosynthesis imaging system can be developed that provides more information on the presence of lesions while maintaining or reducing the dose to the patient. Before this imaging methodology can be translated to routine clinical use, a series of issues and concerns related to tomosynthesis imaging must be addressed. This work investigates the relevant physical processes to improve our understanding and enable the introduction of this tomographic imaging method to the realm of clinical breast imaging. The processes investigated in this work included the dosimetry involved in tomosynthesis imaging, x-ray scatter in the projection images, imaging system performance, and acquisition geometry. A comprehensive understanding of the glandular dose to the breast during tomosynthesis imaging, as well as the dose distribution to most of the radiosensitive tissues in the body from planar mammography, tomosynthesis and dedicated breast computed tomography was gained. The analysis of the behavior of x-ray scatter in tomosynthesis yielded an in-depth characterization of the variation of this effect in the projection images. Finally, the theoretical modeling of a tomosynthesis imaging system, combined with the other results of this work was used to find the geometrical parameters that maximize the quality of the tomosynthesis reconstruction.Ph.D.Andrew Karellas, John N. Oshinski, Xiaoping P. Hu, Carl J. D’Orsi and Ernest V. Garci

Machine learning methods for the analysis and interpretation of images and other multi-dimensional data

L'abstract è presente nell'allegato / the abstract is in the attachmen

Deep learning in breast cancer screening

Breast cancer is the most common cancer form among women worldwide and the incidence is rising. When mammography was introduced in the 1980s, mortality rates decreased by 30% to 40%. Today all women in Sweden between 40 to 74 years are invited to screening every 18 to 24 months. All women attending screening are examined with mammography, using two views, the mediolateral oblique (MLO) view and the craniocaudal (CC) view, producing four images in total. The screening process is the same for all women and based purely on age, and not on other risk factors for developing breast cancer. Although the introduction of population-based breast cancer screening is a great success, there are still problems with interval cancer (IC) and large screen detected cancers (SDC), which are connected to an increased morbidity and mortality. To have a good prognosis, it is important to detect a breast cancer early while it has not spread to the lymph nodes, which usually means that the primary tumor is small. To improve this, we need to individualize the screening program, and be flexible on screening intervals and modalities depending on the individual breast cancer risk and mammographic sensitivity. In Sweden, at present, the only modality in the screening process is mammography, which is excellent for a majority of women but not for all. The major lack of breast radiologists is another problem that is pressing and important to address. As their expertise is in such demand, it is important to use their time as efficiently as possible. This means that they should primarily spend time on difficult cases and less time on easily assessed mammograms and healthy women. One challenge is to determine which women are at high risk of being diagnosed with aggressive breast cancer, to delineate the low-risk group, and to take care of these different groups of women appropriately. In studies II to IV we have analysed how we can address these challenges by using deep learning techniques. In study I, we described the cohort from which the study populations for study II to IV were derived (as well as study populations in other publications from our research group). This cohort was called the Cohort of Screen Aged Women (CSAW) and contains all 499,807 women invited to breast cancer screening within the Stockholm County between 2008 to 2015. We also described the future potentials of the dataset, as well as the case control subset of annotated breast tumors and healthy mammograms. This study was presented orally at the annual meeting of the Radiological Society of North America in 2019. In study II, we analysed how a deep learning risk score (DLrisk score) performs compared with breast density measurements for predicting future breast cancer risk. We found that the odds ratios (OR) and areas under the receiver operating characteristic curve (AUC) were higher for age-adjusted DLrisk score than for dense area and percentage density. The numbers for DLrisk score were: OR 1.56, AUC, 0.65; dense area: OR 1.31, AUC 0.60, percent density: OR 1.18, AUC, 0.57; with P < .001 for differences between all AUCs). Also, the false-negative rates, in terms of missed future cancer, was lower for the DLrisk score: 31%, 36%, and 39% respectively. This difference was most distinct for more aggressive cancers. In study III, we analyzed the potential cancer yield when using a commercial deep learning software for triaging screening examinations into two work streams – a ‘no radiologist’ work stream and an ‘enhanced assessment’ work stream, depending on the output score of the AI tumor detection algorithm. We found that the deep learning algorithm was able to independently declare 60% of all mammograms with the lowest scores as “healthy” without missing any cancer. In the enhanced assessment work stream when including the top 5% of women with the highest AI scores, the potential additional cancer detection rate was 53 (27%) of 200 subsequent IC, and 121 (35%) of 347 next-round screen-detected cancers. In study IV, we analyzed different principles for choosing the threshold for the continuous abnormality score when introducing a deep learning algorithm for assessment of mammograms in a clinical prospective breast cancer screening study. The deep learning algorithm was supposed to act as a third independent reader making binary decisions in a double-reading environment (ScreenTrust CAD). We found that the choice of abnormality threshold will have important consequences. If the aim is to have the algorithm work at the same sensitivity as a single radiologist, a marked increase in abnormal assessments must be accepted (abnormal interpretation rate 12.6%). If the aim is to have the combined readers work at the same sensitivity as before, a lower sensitivity of AI compared to radiologists is the consequence (abnormal interpretation rate 7.0%). This study was presented as a poster at the annual meeting of the Radiological Society of North America in 2021. In conclusion, we have addressed some challenges and possibilities by using deep learning techniques to make breast cancer screening programs more individual and efficient. Given the limitations of retrospective studies, there is a now a need for prospective clinical studies of deep learning in mammography screening

Matching of Mammographic Lesions in Different Breast Projections

De todos os cancros, cancro da mama é o que causa mais mortes entre mulheres. Programas de rastreio do cancro da mama podem ajudar a decrescer esta mortalidade, visto que deteção e tratamento do tumor em fases iniciais aumentam a taxa de sobrevivência. Normalmente, um par de radiologistas fazem a interpretação das mamografias, no entanto o processo é longo e cansativo. Isto incentivou o desenvolvimento de sistemas de diagnósitco auxiliado por computador (CADx), para substituir o segundo radiologista, fazendo melhor uso do tempo de especialistas. No entanto, sistemas CADx são associados a taxas elevadas de falsos positivos, dado que a maior parte detes apenas usam uma vista (craniocaudal ou mediolateral oblique) da mamografia. O radiologista, por sua vez, usa ambas as projeções, baseando o seu diagnóstico em diferenças visíveis entre as duas vistas. Quando se consideram as duas projeções da mamografia, a correspondência de lesões é um passo necessário para se fazer o diagnóstico. No entanto, isto é uma tarefa complexa, dado que podem existir vários candidatos a lesão, em cada uma das vistas, para se fazer correspondência. Neste trabalho, um sistema que faz correspondências entre lesões é proposto. Este é composto por três blocos: detetor de candidatos, extração de caraterísticas e correspondência de lesões. O primeiro é uma replicação do trabalho de Ribli et al., e o seu propósito é detetar possíveis candidatos a lesão. O segundo é a extração de vetores de caraterísticas de cada candidato, quer usando a backbone do detetor de candidatos, quer extraindo caraterísticas mais tradicionais, ou usando uma rede neuronal treinada com a triplet loss para distinguir lesões. O terceiro é o cálculo da distância entre os vetores de caraterísticas, usando também heurísticas para restringir possíveis pares de candidatos incorretos, e a ordenação de distâncias para atribuir a correspondência de cada lesão. Este trabalho oferece várias opções de possíveis extractores de caraterísticas e heurísticas a serem incroporados num sistema CADx que seja baseado em detetores de objetos. O facto do modelo treinado com a triplet loss ser competitivo com os restantos modelos, torna o sistema bastante mais viável, sendo que este oferece a possibilidade de a correspondência ser independente da deteção de candidatos. Heurísticas "hard" e "soft" são introduzidas como métodos para limitar correspondências. O sistema é capaz de fazer correspondências de forma satisfatória, dado que a sua exatidão ( 70%85%) é significativamente maior que a probabilidade aleatória (30%40%) dos dados usados. Heurísticas "hard" têm resultados encorajantes na precision@k, dado que estas rejeitam um número significativo de falsos positivos gerados pelo detetor de lesões.Of all cancer diseases, breast cancer is the most lethal among women. It has been shown that breast cancer screening programs can decrease mortality, since early detection increases the chances of survival. Usually, a pair of radiologists interpret the screening mammograms, however the process is long and exhausting. This has encouraged the development of computer aided diagnosis (CADx) systems to replace the second radiologist, making a better use of human-experts' time. But CADx systems are associated with high false positive rates, since most of them only use one view (craniocaudal or mediolateral oblique) of the screening mammogram. Radiologist, on the other hand, use both views; frequently reasoning about the diagnosis by noticeable differences between the two views. When considering both projections of a mammogram, lesion matching is a necessary step to perform diagnosis. However this is a complex task, since there might be various lesion candidates on both projections to match. In this work, a matching system is proposed. The system is a cascade of three blocks: candidates detector, feature extraction and lesion matching. The first is a replication of Ribli et al.'s Faster R-CNN and its purpose is to find possible lesion candidates. The second is the feature vector extraction of each candidate, either by using the candidates detector's backbone, handcrafted features or a siamese network model trained for distinguish lesions. The third is the calculus of the distance between feature vector, also using some heuristics to restrain possible non-lesion pairs, and the ranking of the distances to match the lesions. This work provides several options of possible feature extractors and heuristics to be incorporated into a CADx system based on object detectors. The fact that the triplet loss trained models obtained competitive results with the other features extractors is valuable, since it offers some independence between the detection and matching tasks. "Hard" heuristics and "soft" heurisitcs are introduced as methods to restrain matching. The system is able to detect matches satisfactorily, since its accuracy (70%85%) is significantly higher than chance level (30%40%). "Hard" heuristics proposals achieved encouraging results on precision@k, due to its match and candidates exclusion methods, which rejects a significant number of false positives generated by the object detector

Studies on deep learning approach in breast lesions detection and cancer diagnosis in mammograms

Breast cancer accounts for the largest proportion of newly diagnosed cancers in women recently. Early diagnosis of breast cancer can improve treatment outcomes and reduce mortality. Mammography is convenient and reliable, which is the most commonly used method for breast cancer screening. However, manual examinations are limited by the cost and experience of radiologists, which introduce a high false positive rate and false examination. Therefore, a high-performance computer-aided diagnosis (CAD) system is significant for lesions detection and cancer diagnosis. Traditional CADs for cancer diagnosis require a large number of features selected manually and remain a high false positive rate. The methods based on deep learning can automatically extract image features through the network, but their performance is limited by the problems of multicenter data biases, the complexity of lesion features, and the high cost of annotations. Therefore, it is necessary to propose a CAD system to improve the ability of lesion detection and cancer diagnosis, which is optimized for the above problems. This thesis aims to utilize deep learning methods to improve the CADs' performance and effectiveness of lesion detection and cancer diagnosis. Starting from the detection of multi-type lesions using deep learning methods based on full consideration of characteristics of mammography, this thesis explores the detection method of microcalcification based on multiscale feature fusion and the detection method of mass based on multi-view enhancing. Then, a classification method based on multi-instance learning is developed, which integrates the detection results from the above methods, to realize the precise lesions detection and cancer diagnosis in mammography. For the detection of microcalcification, a microcalcification detection network named MCDNet is proposed to overcome the problems of multicenter data biases, the low resolution of network inputs, and scale differences between microcalcifications. In MCDNet, Adaptive Image Adjustment mitigates the impact of multicenter biases and maximizes the input effective pixels. Then, the proposed pyramid network with shortcut connections ensures that the feature maps for detection contain more precise localization and classification information about multiscale objects. In the structure, trainable Weighted Feature Fusion is proposed to improve the detection performance of both scale objects by learning the contribution of feature maps in different stages. The experiments show that MCDNet outperforms other methods on robustness and precision. In case the average number of false positives per image is 1, the recall rates of benign and malignant microcalcification are 96.8% and 98.9%, respectively. MCDNet can effectively help radiologists detect microcalcifications in clinical applications. For the detection of breast masses, a weakly supervised multi-view enhancing mass detection network named MVMDNet is proposed to solve the lack of lesion-level labels. MVMDNet can be trained on the image-level labeled dataset and extract the extra localization information by exploring the geometric relation between multi-view mammograms. In Multi-view Enhancing, Spatial Correlation Attention is proposed to extract correspondent location information between different views while Sigmoid Weighted Fusion module fuse diagnostic and auxiliary features to improve the precision of localization. CAM-based Detection module is proposed to provide detections for mass through the classification labels. The results of experiments on both in-house dataset and public dataset, [email protected] and [email protected] (recall rate@average number of false positive per image), demonstrate MVMDNet achieves state-of-art performances among weakly supervised methods and has robust generalization ability to alleviate the multicenter biases. In the study of cancer diagnosis, a breast cancer classification network named CancerDNet based on Multi-instance Learning is proposed. CancerDNet successfully solves the problem that the features of lesions are complex in whole image classification utilizing the lesion detection results from the previous chapters. Whole Case Bag Learning is proposed to combined the features extracted from four-view, which works like a radiologist to realize the classification of each case. Low-capacity Instance Learning and High-capacity Instance Learning successfully integrate the detections of multi-type lesions into the CancerDNet, so that the model can fully consider lesions with complex features in the classification task. CancerDNet achieves the AUC of 0.907 and AUC of 0.925 on the in-house and the public datasets, respectively, which is better than current methods. The results show that CancerDNet achieves a high-performance cancer diagnosis. In the works of the above three parts, this thesis fully considers the characteristics of mammograms and proposes methods based on deep learning for lesions detection and cancer diagnosis. The results of experiments on in-house and public datasets show that the methods proposed in this thesis achieve the state-of-the-art in the microcalcifications detection, masses detection, and the case-level classification of cancer and have a strong ability of multicenter generalization. The results also prove that the methods proposed in this thesis can effectively assist radiologists in making the diagnosis while saving labor costs

Diseases of the Chest, Breast, Heart and Vessels 2019-2022

This open access book focuses on diagnostic and interventional imaging of the chest, breast, heart, and vessels. It consists of a remarkable collection of contributions authored by internationally respected experts, featuring the most recent diagnostic developments and technological advances with a highly didactical approach. The chapters are disease-oriented and cover all the relevant imaging modalities, including standard radiography, CT, nuclear medicine with PET, ultrasound and magnetic resonance imaging, as well as imaging-guided interventions. As such, it presents a comprehensive review of current knowledge on imaging of the heart and chest, as well as thoracic interventions and a selection of "hot topics". The book is intended for radiologists, however, it is also of interest to clinicians in oncology, cardiology, and pulmonology