Interleaved text/image Deep Mining on a large-scale radiology database

Despite tremendous progress in computer vision, effec-tive learning on very large-scale (> 100K patients) medi-cal image databases has been vastly hindered. We present an interleaved text/image deep learning system to extract and mine the semantic interactions of radiology images and reports from a national research hospital’s picture archiv-ing and communication system. Instead of using full 3D medical volumes, we focus on a collection of representa-tive ~216K 2D key images/slices (selected by clinicians for diagnostic reference) with text-driven scalar and vector la-bels. Our system interleaves between unsupervised learn-ing (e.g., latent Dirichlet allocation, recurrent neural net language models) on document- and sentence-level texts to generate semantic labels and supervised learning via deep convolutional neural networks (CNNs) to map from images to label spaces. Disease-related key words can be predicted for radiology images in a retrieval manner. We have demon-strated promising quantitative and qualitative results. The large-scale datasets of extracted key images and their cat-egorization, embedded vector labels and sentence descrip-tions can be harnessed to alleviate the deep learning “data-hungry ” obstacle in the medical domain

Towards Lipidomics of Low-Abundant Species for Exploring Tumor Heterogeneity Guided by High-Resolution Mass Spectrometry Imaging

Many studies have evidenced the main role of lipids in physiological and also pathological processes such as cancer, diabetes or neurodegenerative diseases. The identification and the in situ localization of specific low-abundant lipid species involved in cancer biology are still challenging for both fundamental studies and lipid marker discovery. In this paper, we report the identification and the localization of specific isobaric minor phospholipids in human breast cancer xenografts by FTICR MALDI imaging supported by histochemistry. These potential candidates can be further confirmed by liquid chromatography coupled with electrospray mass spectrometry (LC-ESI-MS) after extraction from the region of interest defined by MALDI imaging. Finally, this study highlights the importance of characterizing the heterogeneous distribution of low-abundant lipid species, relevant in complex histological samples for biological purposes.Peer reviewe

Representación de imágenes de histopatología utilizada en tareas de análisis automático: estado del arte

This paper presents a review of the state-of-the-art in histopathology image representation used in automatic image analysis tasks. Automatic analysis of histopathology images is important for building computer-assisted diagnosis tools, automatic image enhancing systems and virtual microscopy systems, among other applications. Histopathology images have a rich mix of visual patterns with particularities that make them difficult to analyze. The paper discusses these particularities, the acquisition process and the challenges found when doing automatic analysis. Second an overview of recent works and methods addressed to deal with visual content representation in different automatic image analysis tasks is presented. Third an overview of applications of image representation methods in several medical domains and tasks is presented. Finally, the paper concludes with current trends of automatic analysis of histopathology images like digital pathology

Automatic Annotation, Classification and Retrieval of Traumatic Brain Injury CT Images

Ph.DDOCTOR OF PHILOSOPH

Anwendung Massenspektrometrie basierter Technologie zur Entdeckung räumlicher Peptidsignaturen in der Krebsforschung

Cancer is one of the leading causes of death worldwide, within the molecular and structure complexity of tumors are causal factors for disease progression and treatment standards. With the development of molecular biological techniques, physicians could use genetic variation or protein and metabolic expression profile besides histo-morphologicial evaluation to classify more accurate risk assessment and to guide treatment decisions. The biomarker-driven personalized therapies might improve clinical care, avoid unnecessary treatments and reduce the duration and costs for hospital stay. Therefore, there is a strong demand for more reliable molecular biomarker profiles. In this dissertation, a novel technique called imaging mass spectrometry (MADLI-MSI) is used to investigate the potential of spatially resolved peptide signatures (directly from tumor tissue; in situ) for (i) discrimination of subtypes of serous ovarian cancer (HGSOC) and (ii) risk assessment of neuroblastoma. Univariate and multivariate static methods were used to determine associated peptide signatures. Using complementary methods, liquid chromatography-based mass spectrometry the corresponding proteins to the peptides were identified and verified by immunohistology. Consequently, peptide signatures were identified to predict disease recurrence in early-stage HGSOC patients and to distinguish high-risk neuroblastoma patients from other risk groups. These results suggest that the MALDI-MSI technique is a promising analytical method that facilitates diagnosis and treatment decision-making. It has also provided new biological insights into tumor heterogeneity, that could benefit the development of molecular biomarker profiles. The data of this dissertation have been really published in Journal “Cancers (MDPI)” 2020 and 2021.Onkologische Erkrankungen (Krebs) sind weltweit eine der häufigsten Todesursachen. Die molekulare und strukturelle Komplexität von Tumoren sind ursächlich für die Krankheitsprogression und Therapieanspruch. Mit der Entwicklung von neuen molekularbiologischen Verfahren könnten Ärzte neben der histo-morphologischen Bewertung auch genetische Variationen oder Protein- und Metabolit-Expressionsprofile nutzen, um eine genauere Risikobewertung vorzunehmen und die Behandlungsentscheidung zu treffen. Die personalisierten Therapien können die klinische Versorgung verbessern durch Vermeidung unnötiger Behandlungen und verringerte Dauer und Kosten des Krankenhausaufenthalts. Daher besteht ein starker Bedarf an zuverlässigeren molekularen Biomarker Profilen. In dieser Dissertation wird ein neuartiges Verfahren, die sogenannten bildgebenden Massenspektrometrie (MADLI-MSI) eingesetzte um das Potential von räumlich aufgelösten Peptide-Signaturen (direkt aus dem Tumorgewebe; in situ) für (i) die Diskriminierung von Subtypen des serösen Ovarialkarzinom (HGSOC) zu untersuchen und (ii) die Risikoabschätzung des Neuroblastomes. Dabei wurden univariate und multivariate statischer Verfahren eingesetzt, um assoziierten Peptide- Signaturen zu bestimmen. Mittels komplementärer Verfahren, Flüssigkeitschromatographie basierte Massenspektrometrie wurden die korrespondierenden Proteine zu den Peptiden identifiziert und Immunhistologisch verifiziert. Folglich wurden Peptidsignaturen zur Vorhersage des Wiederauftretens der Krankheit bei HGSOC-Patienten im Frühstadium und zur Unterscheidung von Hochrisiko-Neuroblastom Patienten von anderen Risikogruppen identifiziert. Diese Ergebnisse deuten darauf hin, dass die MALDI-MSI-Technik eine vielversprechende Analysemethode ist, die die Diagnose und die Entscheidung über die Behandlung erleichtert. Außerdem hat sie neue biologische Erkenntnisse über die Heterogenität des Tumors geliefert, die der Entwicklung von molekularen Biomarker-Profilen zu Gute kommen könnten. Die Daten dieser Dissertation wurden in der Zeitschrift „Cancers (MDPI)" 2020 und 2021 veröffentlicht

A Bottom-Up Review of Image Analysis Methods for Suspicious Region Detection in Mammograms.

Breast cancer is one of the most common death causes amongst women all over the world. Early detection of breast cancer plays a critical role in increasing the survival rate. Various imaging modalities, such as mammography, breast MRI, ultrasound and thermography, are used to detect breast cancer. Though there is a considerable success with mammography in biomedical imaging, detecting suspicious areas remains a challenge because, due to the manual examination and variations in shape, size, other mass morphological features, mammography accuracy changes with the density of the breast. Furthermore, going through the analysis of many mammograms per day can be a tedious task for radiologists and practitioners. One of the main objectives of biomedical imaging is to provide radiologists and practitioners with tools to help them identify all suspicious regions in a given image. Computer-aided mass detection in mammograms can serve as a second opinion tool to help radiologists avoid running into oversight errors. The scientific community has made much progress in this topic, and several approaches have been proposed along the way. Following a bottom-up narrative, this paper surveys different scientific methodologies and techniques to detect suspicious regions in mammograms spanning from methods based on low-level image features to the most recent novelties in AI-based approaches. Both theoretical and practical grounds are provided across the paper sections to highlight the pros and cons of different methodologies. The paper's main scope is to let readers embark on a journey through a fully comprehensive description of techniques, strategies and datasets on the topic

Optimization algorithms for inference and classification of genetic profiles from undersampled measurements

In this thesis, we tackle three different problems, all related to optimization techniques for inference and classification of genetic profiles. First, we extend the deterministic Non-negative Matrix Factorization (NMF) framework to the probabilistic case (PNMF). We apply the PNMF algorithm to cluster and classify DNA microarrays data. The proposed PNMF is shown to outperform the deterministic NMF and the sparse NMF algorithms in clustering stability and classification accuracy. Second, we propose SMURC: Small-sample MUltivariate Regression with Covariance estimation. Specifically, we consider a high dimension low sample-size multivariate regression problem that accounts for correlation of the response variables. We show that, in this case, the maximum likelihood approach is senseless because the likelihood diverges. We propose a normalization of the likelihood function that guarantees convergence. Simulation results show that SMURC outperforms the regularized likelihood estimator with known covariance matrix and the state-of-the-art sparse Conditional Graphical Gaussian Model (sCGGM). In the third Chapter, we derive a new greedy algorithm that provides an exact sparse solution of the combinatorial l sub zero-optimization problem in an exponentially less computation time. Unlike other greedy approaches, which are only approximations of the exact sparse solution, the proposed greedy approach, called Kernel reconstruction, leads to the exact optimal solution

Representation learning for histopathology image analysis

Abstract. Nowadays, automatic methods for image representation and analysis have been successfully applied in several medical imaging problems leading to the emergence of novel research areas like digital pathology and bioimage informatics. The main challenge of these methods is to deal with the high visual variability of biological structures present in the images, which increases the semantic gap between their visual appearance and their high level meaning. Particularly, the visual variability in histopathology images is also related to the noise added by acquisition stages such as magnification, sectioning and staining, among others. Many efforts have focused on the careful selection of the image representations to capture such variability. This approach requires expert knowledge as well as hand-engineered design to build good feature detectors that represent the relevant visual information. Current approaches in classical computer vision tasks have replaced such design by the inclusion of the image representation as a new learning stage called representation learning. This paradigm has outperformed the state-of-the-art results in many pattern recognition tasks like speech recognition, object detection, and image scene classification. The aim of this research was to explore and define a learning-based histopathology image representation strategy with interpretative capabilities. The main contribution was a novel approach to learn the image representation for cancer detection. The proposed approach learns the representation directly from a Basal-cell carcinoma image collection in an unsupervised way and was extended to extract more complex features from low-level representations. Additionally, this research proposed the digital staining module, a complementary interpretability stage to support diagnosis through a visual identification of discriminant and semantic features. Experimental results showed a performance of 92% in F-Score, improving the state-of-the-art representation by 7%. This research concluded that representation learning improves the feature detectors generalization as well as the performance for the basal cell carcinoma detection task. As additional contributions, a bag of features image representation was extended and evaluated for Alzheimer detection, obtaining 95% in terms of equal error classification rate. Also, a novel perspective to learn morphometric measures in cervical cells based on bag of features was presented and evaluated obtaining promising results to predict nuclei and cytoplasm areas.Los métodos automáticos para la representación y análisis de imágenes se han aplicado con éxito en varios problemas de imagen médica que conducen a la aparición de nuevas áreas de investigación como la patología digital. El principal desafío de estos métodos es hacer frente a la alta variabilidad visual de las estructuras biológicas presentes en las imágenes, lo que aumenta el vacío semántico entre su apariencia visual y su significado de alto nivel. Particularmente, la variabilidad visual en imágenes de histopatología también está relacionada con el ruido añadido por etapas de adquisición tales como magnificación, corte y tinción entre otros. Muchos esfuerzos se han centrado en la selección de la representacion de las imágenes para capturar dicha variabilidad. Este enfoque requiere el conocimiento de expertos y el diseño de ingeniería para construir buenos detectores de características que representen la información visual relevante. Los enfoques actuales en tareas de visión por computador han reemplazado ese diseño por la inclusión de la representación en la etapa de aprendizaje. Este paradigma ha superado los resultados del estado del arte en muchas de las tareas de reconocimiento de patrones tales como el reconocimiento de voz, la detección de objetos y la clasificación de imágenes. El objetivo de esta investigación es explorar y definir una estrategia basada en el aprendizaje de la representación para imágenes histopatológicas con capacidades interpretativas. La contribución principal de este trabajo es un enfoque novedoso para aprender la representación de la imagen para la detección de cáncer. El enfoque propuesto aprende la representación directamente de una colección de imágenes de carcinoma basocelular en forma no supervisada que permite extraer características más complejas a partir de las representaciones de bajo nivel. También se propone el módulo de tinción digital, una nueva etapa de interpretabilidad para apoyar el diagnóstico a través de una identificación visual de las funciones discriminantes y semánticas. Los resultados experimentales mostraron un rendimiento del 92% en términos de F-Score, mejorando la representación del estado del arte en un 7%. Esta investigación concluye que el aprendizaje de la representación mejora la generalización de los detectores de características así como el desempeño en la detección de carcinoma basocelular. Como contribuciones adicionales, una representación de bolsa de caracteristicas (BdC) fue ampliado y evaluado para la detección de la enfermedad de Alzheimer, obteniendo un 95% en términos de EER. Además, una nueva perspectiva para aprender medidas morfométricas en las células del cuello uterino basado en BdC fue presentada y evaluada obteniendo resultados prometedores para predecir las areás del nucleo y el citoplasma.Maestrí