MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network

The inability to interpret the model prediction in semantically and visually meaningful ways is a well-known shortcoming of most existing computer-aided diagnosis methods. In this paper, we propose MDNet to establish a direct multimodal mapping between medical images and diagnostic reports that can read images, generate diagnostic reports, retrieve images by symptom descriptions, and visualize attention, to provide justifications of the network diagnosis process. MDNet includes an image model and a language model. The image model is proposed to enhance multi-scale feature ensembles and utilization efficiency. The language model, integrated with our improved attention mechanism, aims to read and explore discriminative image feature descriptions from reports to learn a direct mapping from sentence words to image pixels. The overall network is trained end-to-end by using our developed optimization strategy. Based on a pathology bladder cancer images and its diagnostic reports (BCIDR) dataset, we conduct sufficient experiments to demonstrate that MDNet outperforms comparative baselines. The proposed image model obtains state-of-the-art performance on two CIFAR datasets as well.Comment: CVPR2017 Ora

Combining handcrafted features with latent variables in machine learning for prediction of radiationâ induced lung damage

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/149351/1/mp13497.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/149351/2/mp13497_am.pd

Automatic design of neuromarkers for obsessive compulsive disorder characterisation

This bacherlor thesis proposes a new paradigm to discover biomarkers capable of characterizing obsessive-compulsive disorder (OCD) by means of machine learning methods. These biomarkers, named neuromarkers, will be obtained through the analysis of sets of magnetic resonance images of the brains of OCD patients and healthy control subjects. The design of the neuromarkers stems from a method for the automatic discovery of clusters of voxels, distributed in separate brain regions, relevant to OCD. This method was recently published by Dr. Emilio Parrado Hernández, Dr. Vanessa Gómez Verdejo and Dr. Manel Martínez Ramón. With these clusters as a starting point, we will de ne the neuromarkers as a set of measurements describing features of these individual regions. Then we will perform a selection of these neuromarkers, using state of the art feature selection techniques, to arrive at a reduced, relevant and intuitive set. The results will be sent to Dr. Carles Soriano Mas at the Bellvitge University Hospital in Barcelona, Spain. His feedback will be used to determine the e cacy of our neuromarkers and their usefulness for psychiatric analysis. The main goal of the project is to come up with a set of neuromarkers for OCD characterisation that are easy to interpret and handle by the psychiatric community. A paper presenting the methods and results described in this bachelor thesis, of which the student is the main author, has been submitted and accepted for presentation in the 2014 European Congress of Machine Learning (ECML/PKDD 2014). The ECML reported a 23.8% paper acceptance rate for 2014.Ingeniería de Sistemas Audiovisuale

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Assessment of immunological features in muscle-invasive bladder cancer prognosis using ensemble learning

Funding: This research received financial support from Definiens GmbH and the Industrial Centre for AI Research in digital Diagnostics (iCAIRD) which is funded by Innovate UK on behalf of UK Research and Innovation (UKRI) [project number: 104690].The clinical staging and prognosis of muscle-invasive bladder cancer (MIBC) routinely includes the assessment of patient tissue samples by a pathologist. Recent studies corroborate the importance of image analysis in identifying and quantifying immunological markers from tissue samples that can provide further insight into patient prognosis. In this paper, we apply multiplex immunofluorescence to MIBC tissue sections to capture whole-slide images and quantify potential prognostic markers related to lymphocytes, macrophages, tumour buds, and PD-L1. We propose a machine-learning-based approach for the prediction of 5 year prognosis with different combinations of image, clinical, and spatial features. An ensemble model comprising several functionally different models successfully stratifies MIBC patients into two risk groups with high statistical significance (p value < 1×10−5). Critical to improving MIBC survival rates, our method correctly classifies 71.4% of the patients who succumb to MIBC, which is significantly more than the 28.6% of the current clinical gold standard, the TNM staging system.Publisher PDFPeer reviewe

Classifying brain metastases by their primary site of origin using a radiomics approach based on texture analysis: a feasibility study

[EN] Objective To examine the capability of MRI texture analysis to differentiate the primary site of origin of brain metastases following a radiomics approach. Methods Sixty-seven untreated brain metastases (BM) were found in 3D T1-weighted MRI of 38 patients with cancer: 27 from lung cancer, 23 from melanoma and 17 from breast cancer. These lesions were segmented in 2D and 3D to compare the discriminative power of 2D and 3D texture features. The images were quantized using different number of gray-levels to test the influence of quantization. Forty-three rotation-invariant texture features were examined. Feature selection and random forest classification were implemented within a nested cross-validation structure. Classification was evaluated with the area under receiver operating characteristic curve (AUC) considering two strategies: multiclass and one-versus-one. Results In the multiclass approach, 3D texture features were more discriminative than 2D features. The best results were achieved for images quantized with 32 gray-levels (AUC = 0.873 +/- 0.064) using the top four features provided by the feature selection method based on the p-value. In the one-versus-one approach, high accuracy was obtained when differentiating lung cancer BM from breast cancer BM (four features, AUC = 0.963 +/- 0.054) and melanoma BM (eight features, AUC = 0.936 +/- 0.070) using the optimal dataset (3D features, 32 gray-levels). Classification of breast cancer and melanoma BM was unsatisfactory (AUC = 0.607 +/- 0.180). Conclusion Volumetric MRI texture features can be useful to differentiate brain metastases from different primary cancers after quantizing the images with the proper number of gray-levels.This work has been partially funded by the Spanish Ministerio de Economia y Competitividad (MINECO) and FEDER funds under Grant BFU2015-64380-C2-2-R. Rafael Ortiz-Ramon was supported by grant ACIF/2015/078 from the Conselleria d'Educacio, Investigacio, Cultura i Esport of the Valencian Community (Spain). Andres Larroza was supported by grant FPU12/01140 from the Spanish Ministerio de Educacion, Cultura y Deporte (MECD).Ortiz-Ramón, R.; Larroza-Santacruz, A.; Ruiz-España, S.; Arana Fernandez De Moya, E.; Moratal, D. (2018). Classifying brain metastases by their primary site of origin using a radiomics approach based on texture analysis: a feasibility study. European Radiology. 28(11):4514-4523. https://doi.org/10.1007/s00330-018-5463-6S451445232811Gavrilovic IT, Posner JB (2005) Brain metastases: epidemiology and pathophysiology. J Neurooncol 75:5–14Stelzer KJ (2013) Epidemiology and prognosis of brain metastases. Surg Neurol Int 4:S192–S202Soffietti R, Cornu P, Delattre JY et al (2006) EFNS Guidelines on diagnosis and treatment of brain metastases: report of an EFNS Task Force. Eur J Neurol 13:674–681Kaal ECA, Taphoorn MJB, Vecht CJ (2005) Symptomatic management and imaging of brain metastases. J Neurooncol 75:15–20Nayak L, Lee EQ, Wen PY (2012) Epidemiology of brain metastases. Curr Oncol Rep 14:48–54Bartelt S, Lutterbach J (2003) Brain metastases in patients with cancer of unknown primary. J Neurooncol 64:249–253Agazzi S, Pampallona S, Pica A et al (2004) The origin of brain metastases in patients with an undiagnosed primary tumor. Acta Neurochir (Wien) 146:153–157Pekmezci M, Perry A (2013) Neuropathology of brain metastases. Surg Neurol Int 4:245Zakaria R, Das K, Bhojak M et al (2014) The role of magnetic resonance imaging in the management of brain metastases: diagnosis to prognosis. Cancer Imaging 14:1–8Bekaert L, Emery E, Levallet G, Lechapt-Zalcman E (2017) Histopathologic diagnosis of brain metastases: current trends in management and future considerations. Brain Tumor Pathol 34:8–19Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: images are more than pictures, they are data. Radiology 278:563–577Lambin P, Rios-Velazquez E, Leijenaar R et al (2012) Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 48:441–446Yip SSF, Aerts HJWL (2016) Applications and limitations of radiomics. Phys Med Biol 61:R150–R166Kumar V, Gu Y, Basu S et al (2012) Radiomics: the process and the challenges. Magn Reson Imaging 30:1234–1248Castellano G, Bonilha L, Li LM, Cendes F (2004) Texture analysis of medical images. Clin Radiol 59:1061–1069Kassner A, Thornhill RE (2010) Texture analysis: a review of neurologic MR imaging applications. AJNR Am J Neuroradiol 31:809–816Mahmoud-Ghoneim D, Toussaint G, Constans JM, De Certaines JD (2003) Three dimensional texture analysis in MRI: a preliminary evaluation in gliomas. Magn Reson Imaging 21:983–987Fetit AE, Novak J, Peet AC, Arvanitis TN (2015) Three-dimensional textural features of conventional MRI improve diagnostic classification of childhood brain tumors. NMR Biomed 28:1174–1184Zacharaki EI, Wang S, Chawla S et al (2009) Classification of brain tumor type and grade using MRI texture and shape in a machine learning scheme. Magn Reson Med 62:1609–1618Georgiadis P, Cavouras D, Kalatzis I et al (2009) Enhancing the discrimination accuracy between metastases, gliomas and meningiomas on brain MRI by volumetric textural features and ensemble pattern recognition methods. Magn Reson Imaging 27:120–130Larroza A, Moratal D, Paredes-Sánchez A et al (2015) Support vector machine classification of brain metastasis and radiation necrosis based on texture analysis in MRI. J Magn Reson Imaging 42:1362–1368Li Z, Mao Y, Li H et al (2016) Differentiating brain metastases from different pathological types of lung cancers using texture analysis of T1 postcontrast MR. Magn Reson Med 76:1410–1419Fink KR, Fink JR (2013) Imaging of brain metastases. Surg Neurol Int 4:S209–S219Larroza A, Bodí V, Moratal D (2016) Texture analysis in magnetic resonance imaging: review and considerations for future applications. In: Assessment of cellular and organ function and dysfunction using direct and derived MRI methodologies. InTech, Rijeka, Croatia, pp 75–106Leite M, Rittner L, Appenzeller S et al (2015) Etiology-based classification of brain white matter hyperintensity on magnetic resonance imaging. J Med Imaging 2:14002Mahmoud-Ghoneim D, Alkaabi MK, De Certaines JD, Goettsche F-M (2008) The impact of image dynamic range on texture classification of brain white matter. BMC Med Imaging 8:1–8Depeursinge A, Foncubierta-Rodriguez A, Van De Ville D, Müller H (2014) Three-dimensional solid texture analysis in biomedical imaging: review and opportunities. Med Image Anal 18:176–196Ellingson BM, Bendszus M, Boxerman J et al (2015) Consensus recommendations for a standardized Brain Tumor Imaging Protocol in clinical trials. Neuro Oncol 17:1188–1198Mayerhoefer ME, Breitenseher MJ, Kramer J et al (2005) Texture analysis for tissue discrimination on T1-weighted MR images of the knee joint in a multicenter study: Transferability of texture features and comparison of feature selection methods and classifiers. J Magn Reson Imaging 22:674–680Waugh SA, Lerski RA, Bidaut L, Thompson AM (2011) The influence of field strength and different clinical breast MRI protocols on the outcome of texture analysis using foam phantoms. Med Phys 38:5058–5066Chan TF, Vese LA (2001) Active contours without edges. IEEE Trans Image Process 10:266–277Collewet G, Strzelecki M, Mariette F (2004) Influence of MRI acquisition protocols and image intensity normalization methods on texture classification. Magn Reson Imaging 22:81–91Gibbs P, Turnbull LW (2003) Textural analysis of contrast-enhanced MR images of the breast. Magn Reson Med 50:92–98Vallières M, Freeman CR, Skamene SR, El Naqa I (2015) A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities. Phys Med Biol 60:5471–5496Kuhn M, Johnson K (2013) Data pre-processing. In: Applied predictive modeling, 1st ed. Springer, New York, NY, pp 27–59Fernández-Delgado M, Cernadas E, Barro S et al (2014) Do we need hundreds of classifiers to solve real world classification problems? J Mach Learn Res 15:3133–3181Caruana R, Karampatziakis N, Yessenalina A (2008) An empirical evaluation of supervised learning in high dimensions. In: Proceedings of the 25th international conference on Machine learning - ICML ’08. ACM Press, Helsinki, Finland, pp 96–103Kuhn M, Johnson K (2013) Over-fitting and model tuning. In: Applied predictive modeling, 1st ed. Springer, New York, NY, pp 61–92Kuhn M, Johnson K (2013) An introduction to feature selection. In: Applied predictive modeling, 1st ed. Springer, New York, NY, pp 487–519Ambroise C, McLachlan GJ (2002) Selection bias in gene extraction on the basis of microarray gene-expression data. Proc Natl Acad Sci U S A 99:6562–6566Provost F, Domingos P (2003) Tree induction for probability-based ranking. Mach Learn 52:199–215Kuhn M (2008) Building predictive models in R using the caret package. J Stat Softw 28:1–26Ortiz-Ramon R, Larroza A, Arana E, Moratal D (2017) Identifying the primary site of origin of MRI brain metastases from lung and breast cancer following a 2D radiomics approach. In: 2017 I.E. 14th International Symposium on Biomedical Imaging (ISBI 2017). Melbourne, VIC, pp 1213–1216Ortiz-Ramon R, Larroza A, Arana E, Moratal D (2017) A radiomics evaluation of 2D and 3D MRI texture features to classify brain metastases from lung cancer and melanoma. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Seogwipo, pp 493–496Béresová M, Larroza A, Arana E, et al (2017) 2D and 3D texture analysis to differentiate brain metastases on MR images: proceed with caution. MAGMA 1–10Ahmed A, Gibbs P, Pickles M, Turnbull L (2013) Texture analysis in assessment and prediction of chemotherapy response in breast cancer. J Magn Reson Imaging 38:89–101Chen W, Giger ML, Li H et al (2007) Volumetric texture analysis of breast lesions on contrast-enhanced magnetic resonance images. Magn Reson Med 58:562–57

Multiple Instance Learning: A Survey of Problem Characteristics and Applications

Multiple instance learning (MIL) is a form of weakly supervised learning where training instances are arranged in sets, called bags, and a label is provided for the entire bag. This formulation is gaining interest because it naturally fits various problems and allows to leverage weakly labeled data. Consequently, it has been used in diverse application fields such as computer vision and document classification. However, learning from bags raises important challenges that are unique to MIL. This paper provides a comprehensive survey of the characteristics which define and differentiate the types of MIL problems. Until now, these problem characteristics have not been formally identified and described. As a result, the variations in performance of MIL algorithms from one data set to another are difficult to explain. In this paper, MIL problem characteristics are grouped into four broad categories: the composition of the bags, the types of data distribution, the ambiguity of instance labels, and the task to be performed. Methods specialized to address each category are reviewed. Then, the extent to which these characteristics manifest themselves in key MIL application areas are described. Finally, experiments are conducted to compare the performance of 16 state-of-the-art MIL methods on selected problem characteristics. This paper provides insight on how the problem characteristics affect MIL algorithms, recommendations for future benchmarking and promising avenues for research

Advancing Artificial Intelligence in Sensors, Signals, and Imaging Informatics.

ObjectiveTo identify research works that exemplify recent developments in the field of sensors, signals, and imaging informatics.MethodA broad literature search was conducted using PubMed and Web of Science, supplemented with individual papers that were nominated by section editors. A predefined query made from a combination of Medical Subject Heading (MeSH) terms and keywords were used to search both sources. Section editors then filtered the entire set of retrieved papers with each paper having been reviewed by two section editors. Papers were assessed on a three-point Likert scale by two section editors, rated from 0 (do not include) to 2 (should be included). Only papers with a combined score of 2 or above were considered.ResultsA search for papers was executed at the start of January 2019, resulting in a combined set of 1,459 records published in 2018 in 119 unique journals. Section editors jointly filtered the list of candidates down to 14 nominations. The 14 candidate best papers were then ranked by a group of eight external reviewers. Four papers, representing different international groups and journals, were selected as the best papers by consensus of the International Medical Informatics Association (IMIA) Yearbook editorial board.ConclusionsThe fields of sensors, signals, and imaging informatics have rapidly evolved with the application of novel artificial intelligence/machine learning techniques. Studies have been able to discover hidden patterns and integrate different types of data towards improving diagnostic accuracy and patient outcomes. However, the quality of papers varied widely without clear reporting standards for these types of models. Nevertheless, a number of papers have demonstrated useful techniques to improve the generalizability, interpretability, and reproducibility of increasingly sophisticated models