Tumor cell load and heterogeneity estimation from diffusion-weighted MRI calibrated with histological data: an example from lung cancer

Producción CientíficaDiffusion-weighted magnetic resonance imaging (DWI) is a key non-invasive imaging technique for cancer diagnosis and tumor treatment assessment, reflecting Brownian movement of water molecules in tissues. Since densely packed cells restrict molecule mobility, tumor tissues produce usually higher signal (a.k.a. less attenuated signal) on isotropic maps compared with normal tissues. However, no general quantitative relation between DWI data and the cell density has been established. In order to link low-resolution clinical cross-sectional data with high-resolution histological information, we developed an image processing and analysis chain, which was used to study the correlation between the diffusion coefficient (D value) estimated from DWI and tumor cellularity from serial histological slides of a resected non-small cell lung cancer tumor. Color deconvolution followed by cell nuclei segmentation was performed on digitized histological images to determine local and cell-type specific 2d (two-dimensional) densities. From these, the 3d cell density was inferred by a model-based sampling technique, which is necessary for the calculation of local and global 3d tumor cell count. Next, DWI sequence information was overlaid with high-resolution CT data and the resected histology using prominent anatomical hallmarks for co-registration of histology tissue blocks and non-invasive imaging modalities' data. The integration of cell numbers information and DWI data derived from different tumor areas revealed a clear negative correlation between cell density and D value. Importantly, spatial tumor cell density can be calculated based on DWI data. In summary, our results demonstrate that tumor cell count and heterogeneity can be predicted from DWI data, which may open new opportunities for personalized diagnosis and therapy optimization

A survey on artificial intelligence in histopathology image analysis

The increasing adoption of the whole slide image (WSI) technology in histopathology has dramatically transformed pathologists' workflow and allowed the use of computer systems in histopathology analysis. Extensive research in Artificial Intelligence (AI) with a huge progress has been conducted resulting in efficient, effective, and robust algorithms for several applications including cancer diagnosis, prognosis, and treatment. These algorithms offer highly accurate predictions but lack transparency, understandability, and actionability. Thus, explainable artificial intelligence (XAI) techniques are needed not only to understand the mechanism behind the decisions made by AI methods and increase user trust but also to broaden the use of AI algorithms in the clinical setting. From the survey of over 150 papers, we explore different AI algorithms that have been applied and contributed to the histopathology image analysis workflow. We first address the workflow of the histopathological process. We present an overview of various learning-based, XAI, and actionable techniques relevant to deep learning methods in histopathological imaging. We also address the evaluation of XAI methods and the need to ensure their reliability on the field

The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis

Recently, deep learning frameworks have rapidly become the main methodology for analyzing medical images. Due to their powerful learning ability and advantages in dealing with complex patterns, deep learning algorithms are ideal for image analysis challenges, particularly in the field of digital pathology. The variety of image analysis tasks in the context of deep learning includes classification (e.g., healthy vs. cancerous tissue), detection (e.g., lymphocytes and mitosis counting), and segmentation (e.g., nuclei and glands segmentation). The majority of recent machine learning methods in digital pathology have a pre- and/or post-processing stage which is integrated with a deep neural network. These stages, based on traditional image processing methods, are employed to make the subsequent classification, detection, or segmentation problem easier to solve. Several studies have shown how the integration of pre- and post-processing methods within a deep learning pipeline can further increase the model's performance when compared to the network by itself. The aim of this review is to provide an overview on the types of methods that are used within deep learning frameworks either to optimally prepare the input (pre-processing) or to improve the results of the network output (post-processing), focusing on digital pathology image analysis. Many of the techniques presented here, especially the post-processing methods, are not limited to digital pathology but can be extended to almost any image analysis field

The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis.

Recently, deep learning frameworks have rapidly become the main methodology for analyzing medical images. Due to their powerful learning ability and advantages in dealing with complex patterns, deep learning algorithms are ideal for image analysis challenges, particularly in the field of digital pathology. The variety of image analysis tasks in the context of deep learning includes classification (e.g., healthy vs. cancerous tissue), detection (e.g., lymphocytes and mitosis counting), and segmentation (e.g., nuclei and glands segmentation). The majority of recent machine learning methods in digital pathology have a pre- and/or post-processing stage which is integrated with a deep neural network. These stages, based on traditional image processing methods, are employed to make the subsequent classification, detection, or segmentation problem easier to solve. Several studies have shown how the integration of pre- and post-processing methods within a deep learning pipeline can further increase the model's performance when compared to the network by itself. The aim of this review is to provide an overview on the types of methods that are used within deep learning frameworks either to optimally prepare the input (pre-processing) or to improve the results of the network output (post-processing), focusing on digital pathology image analysis. Many of the techniques presented here, especially the post-processing methods, are not limited to digital pathology but can be extended to almost any image analysis field

Nuclei & Glands Instance Segmentation in Histology Images: A Narrative Review

Instance segmentation of nuclei and glands in the histology images is an important step in computational pathology workflow for cancer diagnosis, treatment planning and survival analysis. With the advent of modern hardware, the recent availability of large-scale quality public datasets and the community organized grand challenges have seen a surge in automated methods focusing on domain specific challenges, which is pivotal for technology advancements and clinical translation. In this survey, 126 papers illustrating the AI based methods for nuclei and glands instance segmentation published in the last five years (2017-2022) are deeply analyzed, the limitations of current approaches and the open challenges are discussed. Moreover, the potential future research direction is presented and the contribution of state-of-the-art methods is summarized. Further, a generalized summary of publicly available datasets and a detailed insights on the grand challenges illustrating the top performing methods specific to each challenge is also provided. Besides, we intended to give the reader current state of existing research and pointers to the future directions in developing methods that can be used in clinical practice enabling improved diagnosis, grading, prognosis, and treatment planning of cancer. To the best of our knowledge, no previous work has reviewed the instance segmentation in histology images focusing towards this direction.Comment: 60 pages, 14 figure

Nuclei/Cell Detection in Microscopic Skeletal Muscle Fiber Images and Histopathological Brain Tumor Images Using Sparse Optimizations

Nuclei/Cell detection is usually a prerequisite procedure in many computer-aided biomedical image analysis tasks. In this thesis we propose two automatic nuclei/cell detection frameworks. One is for nuclei detection in skeletal muscle fiber images and the other is for brain tumor histopathological images. For skeletal muscle fiber images, the major challenges include: i) shape and size variations of the nuclei, ii) overlapping nuclear clumps, and iii) a series of z-stack images with out-of-focus regions. We propose a novel automatic detection algorithm consisting of the following components: 1) The original z-stack images are first converted into one all-in-focus image. 2) A sufficient number of hypothetical ellipses are then generated for each nuclei contour. 3) Next, a set of representative training samples and discriminative features are selected by a two-stage sparse model. 4) A classifier is trained using the refined training data. 5) Final nuclei detection is obtained by mean-shift clustering based on inner distance. The proposed method was tested on a set of images containing over 1500 nuclei. The results outperform the current state-of-the-art approaches. For brain tumor histopathological images, the major challenges are to handle significant variations in cell appearance and to split touching cells. The proposed novel automatic cell detection consists of: 1) Sparse reconstruction for splitting touching cells. 2) Adaptive dictionary learning for handling cell appearance variations. The proposed method was extensively tested on a data set with over 2000 cells. The result outperforms other state-of-the-art algorithms with F1 score = 0.96

Multimodal FTIR Microscopy-guided Acquisition and Interpretation of MALDI Mass Spectrometry Imaging Data

Multimodale klinische Bildgebung stellt eine der bedeutendsten Entwicklung der letzten Jahrzehnte dar. Neben der Kombination komplementärer in vivo Sensoren in beispielsweise PET-MRI oder SPECT-CT sind auch ex vivo Analyseverfahren, welche eine genauere Beschreibung der Probe ermöglichen, in den Bereich der (prä)klinischen Diagnostik vorgedrungen. Eine der vielversprechendsten Techniken in diesem Zusammenhang stellt die bildgebende Massenspektrometrie dar, welche die Verteilungsmuster hunderter Biomoleküle oder Pharmazeutika semi-quantitativ erfasst. Dabei kommt das Verfahren ohne die Verwendung von markierten Substanzen aus und erlaubt eine höhere räumliche und spektrale Auflösung im Vergleich zu in vivo Sensoren. Allerdings unterliegt die Technik auch einigen wesentlichen Einschränkungen, da die Datenakquisition besonders bei der Verwendung von ultrahochauflösenden FTICR-Detektoren sehr langsam erfolgt. Die niedrige Durchsatzleistung und damit verbundene unhandliche Datenmenge erschwert somit die Analyse größerer Patientenkohorten, wodurch ein Bedarf an multimodalen Lösungsansätzen besteht. Ein geeignetes Verfahren in dieser Hinsicht stellt die Schwingungsspektroskopie (bsp. Infrarotspektroskopie) dar, welche räumliche Details vergleichsweise schnell erfasst; dabei allerdings keine Rückschlüsse auf die Verteilung bestimmter chemischer Substanzen ermöglicht. Im Rahmen der vorliegenden Arbeit wurde ein MATLAB-gestütztes Verfahren zur multimodalen Akquirierung von Infrarotspektroskopie- und Massenspektrometrie-Daten entwickelt und bewertet. Dabei werden räumliche Strukturen und Zellpopulationen innerhalb von Geweben mittels FTIR-basierter Clusteranalyse segmentiert. Anschließend kann die chemische Zusammensetzung einzelner Segmente zielgerichtet akquiriert und verglichen werden. Das entwickelte Verfahren funktioniert dabei unabhängig von konventioneller histopathologischer Gewebeannotation. Ein wichtiger Faktor bei Mittelinfrarot- und Massenspektrometrie-Messungen auf Gewebe stellt die Zusammensetzung der verwendeten Objektträger-Beschichtung dar. Für die Bewertung der erhaltenen Spektren und der damit verbundenen Bildsegmentierung wurden deshalb Experimente auf Indiumzinnoxid, Silberzinnoxid und Gold durchgeführt und verglichen. Dabei konnte gezeigt werden, dass Infrarot- und Massenspektrometrie-Bilder von der gleichen Probe auf Gold mit hoher Qualität aufgenommen werden können. Weiterhin konnte gezeigt werden, dass durch einfache Infrarotsegmentierung eine Identifizierung relevanter morphologischer Gehirnstrukturen möglich ist. Die erzielte räumliche Präzision und Auflösung der Infrarot-Segmente stellt dabei einen deutlichen Mehrwert gegenüber der direkten Segmentierung von Massenspektrometriebildern dar. Darüber hinaus können Infrarotsegmente bereits vor der eigentlichen MS-Messung generiert werden. Nach erfolgter Methodenentwicklung und Validierung konnte diese auf verschiedene diagnostische Studien angewendet werden. In einem ersten Anwendungsbeispiel konnten in Mäuse xenotransplantierte humane Glioblastomzellen mit erhöhter Präzision visualisiert werden. Darüber hinaus wurde eine im korrespondierenden H&E-Bild unauffällige, den Tumor-umschließende Struktur identifiziert. Durch den erfolgreichen Transfer der Infrarotsegmente in das Koordinatensystem von nachfolgend gemessenen MS-Bildern, konnten spezifische Markersignaturen automatisch extrahiert werden. Im Zuge dessen konnte die Authentizität Tumorstruktur sowie der zweiten Tumor-assoziierten Struktur durch spezifische Massen bekräftigt werden. In einer weiteren Studie, wurde die entwickelte Methode für das automatische Screening von Markersignaturen in Niemann-Pick Typ C1 ähnlichen murinen Kleinhirnschnitten getestet. Dabei konnten regionsspezifische, im Gesamtdatensatz insignifikante Änderungen in der Lipidzusammensetzung automatisiert uns Annotations-unabhängig erfasst werden. In einer weiteren Infrarotspektroskopie-Studie an 89 kryokonservierten GIST Schnitten von 27 Patienten konnte eine schnelle und simultane Segmentierung aller Gewebeproben exemplarisch gezeigt werden. Dabei wurden farbkodierte Bilder aller Proben generiert, in denen gleiche Farben für eine spektrale Ähnlichkeit stehen. Durch den Abgleich der erhaltenen Farbcodes mit histopathologisch annotierten Folgeschnitten konnten zwei der fünf dargestellten Farbgruppen mit dem Auftreten von Tumorzellen assoziiert werden. Die anderen Gruppen repräsentierten Fibrosen, Nekrosen und weitere nicht-tumoröse Gewebeanteile. Abschließend wurde die Struktur-gerichtete Akquisition von ultrahochauflösenden FTICR-MS Bildern gezeigt, welche auf Basis von Mittelinfrarotbildern der identischen Gewebeprobe abgeleitet wurden. Indem die zeitaufwändige MS-Messung ausschließlich auf kleinere Strukturen von Interesse (wie beispielsweise die Körnerzell-Schicht der Cornu Ammonis) gerichtet wurde, konnte eine Zeit- und Datenersparnis von bis zu 97.8% gegenüber der vollständigen Messung erreicht werden. Damit ist ein großer Schritt hin zur Implementierung von ultrahochauflösender Massenspektrometrie im klinischen Umfeld erfolgt

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201