Generating semantically enriched diagnostics for radiological images using machine learning

Development of Computer Aided Diagnostic (CAD) tools to aid radiologists in pathology detection and decision making relies considerably on manually annotated images. With the advancement of deep learning techniques for CAD development, these expert annotations no longer need to be hand-crafted, however, deep learning algorithms require large amounts of data in order to generalise well. One way in which to access large volumes of expert-annotated data is through radiological exams consisting of images and reports. Using past radiological exams obtained from hospital archiving systems has many advantages: they are expert annotations available in large quantities, covering a population-representative variety of pathologies, and they provide additional context to pathology diagnoses, such as anatomical location and severity. Learning to auto-generate such reports from images presents many challenges such as the difficulty in representing and generating long, unstructured textual information, accounting for spelling errors and repetition or redundancy, and the inconsistency across different annotators. In this thesis, the problem of learning to automate disease detection from radiological exams is approached from three directions. Firstly, a report generation model is developed such that it is conditioned on radiological image features. Secondly, a number of approaches are explored aimed at extracting diagnostic information from free-text reports. Finally, an alternative approach to image latent space learning from current state-of-the-art is developed that can be applied to accelerated image acquisition.Open Acces

MammoApplet: an interactive Java applet tool for manual annotation in medical imaging

Web-based applications in computational medicine have become increasingly important during the last years. The rapid growth of the World Wide Web supposes a new paradigm in the telemedicine and eHealth areas in order to assist and enhance the prevention, diagnosis and treatment of patients. Furthermore, training of radiologists and management of medical databases are also becoming increasingly important issues in the field. In this paper, we present MammoApplet , an interactive Java applet interface designed as a web-based tool. It aims to facilitate the diagnosis of new mammographic cases by providing a set of image processing tools that allow a better visualization of the images, and a set of drawing tools, used to annotate the suspicious regions. Each annotation allows including the attributes considered by the experts when issuing the final diagnosis. The overall set of overlays is stored in a database as XML files associated with the original images. The final goal is to obtain a database of already diagnosed cases for training and enhancing the performance of novice radiologistsPeer ReviewedPostprint (author's final draft

Deep Lesion Graphs in the Wild: Relationship Learning and Organization of Significant Radiology Image Findings in a Diverse Large-scale Lesion Database

Radiologists in their daily work routinely find and annotate significant abnormalities on a large number of radiology images. Such abnormalities, or lesions, have collected over years and stored in hospitals' picture archiving and communication systems. However, they are basically unsorted and lack semantic annotations like type and location. In this paper, we aim to organize and explore them by learning a deep feature representation for each lesion. A large-scale and comprehensive dataset, DeepLesion, is introduced for this task. DeepLesion contains bounding boxes and size measurements of over 32K lesions. To model their similarity relationship, we leverage multiple supervision information including types, self-supervised location coordinates and sizes. They require little manual annotation effort but describe useful attributes of the lesions. Then, a triplet network is utilized to learn lesion embeddings with a sequential sampling strategy to depict their hierarchical similarity structure. Experiments show promising qualitative and quantitative results on lesion retrieval, clustering, and classification. The learned embeddings can be further employed to build a lesion graph for various clinically useful applications. We propose algorithms for intra-patient lesion matching and missing annotation mining. Experimental results validate their effectiveness.Comment: Accepted by CVPR2018. DeepLesion url adde

Next Generation Reporting and Diagnostic Tools for Healthcare and Biomedical Applications

Ph.DDOCTOR OF PHILOSOPH

PadChest: A large chest x-ray image dataset with multi-label annotated reports

We present a labeled large-scale, high resolution chest x-ray dataset for the automated exploration of medical images along with their associated reports. This dataset includes more than 160,000 images obtained from 67,000 patients that were interpreted and reported by radiologists at Hospital San Juan Hospital (Spain) from 2009 to 2017, covering six different position views and additional information on image acquisition and patient demography. The reports were labeled with 174 different radiographic findings, 19 differential diagnoses and 104 anatomic locations organized as a hierarchical taxonomy and mapped onto standard Unified Medical Language System (UMLS) terminology. Of these reports, 27% were manually annotated by trained physicians and the remaining set was labeled using a supervised method based on a recurrent neural network with attention mechanisms. The labels generated were then validated in an independent test set achieving a 0.93 Micro-F1 score. To the best of our knowledge, this is one of the largest public chest x-ray database suitable for training supervised models concerning radiographs, and the first to contain radiographic reports in Spanish. The PadChest dataset can be downloaded from http://bimcv.cipf.es/bimcv-projects/padchest/

On the Use of XML in Medical Imaging Web-Based Applications

The rapid growth of digital technology in medical fields over recent years has increased the need for applications able to manage patient medical records, imaging data, and chart information. Web-based applications are implemented with the purpose to link digital databases, storage and transmission protocols, management of large volumes of data and security concepts, allowing the possibility to read, analyze, and even diagnose remotely from the medical center where the information was acquired. The objective of this paper is to analyze the use of the Extensible Markup Language (XML) language in web-based applications that aid in diagnosis or treatment of patients, considering how this protocol allows indexing and exchanging the huge amount of information associated with each medical case. The purpose of this paper is to point out the main advantages and drawbacks of the XML technology in order to provide key ideas for future web-based applicationsPeer ReviewedPostprint (author's final draft