Nuclei & Glands Instance Segmentation in Histology Images: A Narrative Review

Instance segmentation of nuclei and glands in the histology images is an important step in computational pathology workflow for cancer diagnosis, treatment planning and survival analysis. With the advent of modern hardware, the recent availability of large-scale quality public datasets and the community organized grand challenges have seen a surge in automated methods focusing on domain specific challenges, which is pivotal for technology advancements and clinical translation. In this survey, 126 papers illustrating the AI based methods for nuclei and glands instance segmentation published in the last five years (2017-2022) are deeply analyzed, the limitations of current approaches and the open challenges are discussed. Moreover, the potential future research direction is presented and the contribution of state-of-the-art methods is summarized. Further, a generalized summary of publicly available datasets and a detailed insights on the grand challenges illustrating the top performing methods specific to each challenge is also provided. Besides, we intended to give the reader current state of existing research and pointers to the future directions in developing methods that can be used in clinical practice enabling improved diagnosis, grading, prognosis, and treatment planning of cancer. To the best of our knowledge, no previous work has reviewed the instance segmentation in histology images focusing towards this direction.Comment: 60 pages, 14 figure

In vivo imaging reveals transient microglia recruitment and functional recovery of photoreceptor signaling after injury.

Microglia respond to damage and microenvironmental changes within the central nervous system by morphologically transforming and migrating to the lesion, but the real-time behavior of populations of these resident immune cells and the neurons they support have seldom been observed simultaneously. Here, we have used in vivo high-resolution optical coherence tomography (OCT) and scanning laser ophthalmoscopy with and without adaptive optics to quantify the 3D distribution and dynamics of microglia in the living retina before and after local damage to photoreceptors. Following photoreceptor injury, microglia migrated both laterally and vertically through the retina over many hours, forming a tight cluster within the area of visible damage that resolved over 2 wk. In vivo OCT optophysiological assessment revealed that the photoreceptors occupying the damaged region lost all light-driven signaling during the period of microglia recruitment. Remarkably, photoreceptors recovered function to near-baseline levels after the microglia had departed the injury locus. These results demonstrate the spatiotemporal dynamics of microglia engagement and restoration of neuronal function during tissue remodeling and highlight the need for mechanistic studies that consider the temporal and structural dynamics of neuron-microglia interactions in vivo

Probing Cellular Uptake of Nanoparticles, One at a Time

Advanced fluorescence microscopy is the method of choice to study cellular uptake of nanoparticles with molecular specificity and nanoscale resolution; yet, direct visualization of nanoparticles entry into cells poses severe technical challenges. Here, we have combined super-resolution photoactivation localization microscopy (PALM) with single particle tracking (SPT) to visualize clathrin-mediated endocytosis (CME) of polystyrene nanoparticles at very high spatial and temporal resolution

Histopathological image analysis : a review

Over the past decade, dramatic increases in computational power and improvement in image analysis algorithms have allowed the development of powerful computer-assisted analytical approaches to radiological data. With the recent advent of whole slide digital scanners, tissue histopathology slides can now be digitized and stored in digital image form. Consequently, digitized tissue histopathology has now become amenable to the application of computerized image analysis and machine learning techniques. Analogous to the role of computer-assisted diagnosis (CAD) algorithms in medical imaging to complement the opinion of a radiologist, CAD algorithms have begun to be developed for disease detection, diagnosis, and prognosis prediction to complement the opinion of the pathologist. In this paper, we review the recent state of the art CAD technology for digitized histopathology. This paper also briefly describes the development and application of novel image analysis technology for a few specific histopathology related problems being pursued in the United States and Europe

Towards accurate and efficient live cell imaging data analysis

Dynamische zelluläre Prozesse wie Zellzyklus, Signaltransduktion oder Transkription zu analysieren wird Live-cell-imaging mittels Zeitraffermikroskopie verwendet. Um nun aber Zellabstammungsbäume aus einem Zeitraffervideo zu extrahieren, müssen die Zellen segmentiert und verfolgt werden können. Besonders hier, wo lebende Zellen über einen langen Zeitraum betrachtet werden, sind Fehler in der Analyse fatal: Selbst eine extrem niedrige Fehlerrate kann sich amplifizieren, wenn viele Zeitpunkte aufgenommen werden, und damit den gesamten Datensatz unbrauchbar machen. In dieser Arbeit verwenden wir einen einfachen aber praktischen Ansatz, der die Vorzüge der manuellen und automatischen Ansätze kombiniert. Das von uns entwickelte Live-cell-Imaging Datenanalysetool ‘eDetect’ ergänzt die automatische Zellsegmentierung und -verfolgung durch Nachbearbeitung. Das Besondere an dieser Arbeit ist, dass sie mehrere interaktive Datenvisualisierungsmodule verwendet, um den Benutzer zu führen und zu unterstützen. Dies erlaubt den gesamten manuellen Eingriffsprozess zu rational und effizient zu gestalten. Insbesondere werden zwei Streudiagramme und eine Heatmap verwendet, um die Merkmale einzelner Zellen interaktiv zu visualisieren. Die Streudiagramme positionieren ähnliche Objekte in unmittelbarer Nähe. So kann eine große Gruppe ähnlicher Fehler mit wenigen Mausklicks erkannt und korrigiert werden, und damit die manuellen Eingriffe auf ein Minimum reduziert werden. Die Heatmap ist darauf ausgerichtet, alle übersehenen Fehler aufzudecken und den Benutzern dabei zu helfen, bei der Zellabstammungsrekonstruktion schrittweise die perfekte Genauigkeit zu erreichen. Die quantitative Auswertung zeigt, dass eDetect die Genauigkeit der Nachverfolgung innerhalb eines akzeptablen Zeitfensters erheblich verbessern kann. Beurteilt nach biologisch relevanten Metriken, übertrifft die Leistung von eDetect die derer Tools, die den Wettbewerb ‘Cell Tracking Challenge’ gewonnen haben.Live cell imaging based on time-lapse microscopy has been used to study dynamic cellular behaviors, such as cell cycle, cell signaling and transcription. Extracting cell lineage trees out of a time-lapse video requires cell segmentation and cell tracking. For long term live cell imaging, data analysis errors are particularly fatal. Even an extremely low error rate could potentially be amplified by the large number of sampled time points and render the entire video useless. In this work, we adopt a straightforward but practical design that combines the merits of manual and automatic approaches. We present a live cell imaging data analysis tool `eDetect', which uses post-editing to complement automatic segmentation and tracking. What makes this work special is that eDetect employs multiple interactive data visualization modules to guide and assist users, making the error detection and correction procedure rational and efficient. Specifically, two scatter plots and a heat map are used to interactively visualize single cells' visual features. The scatter plots position similar results in close vicinity, making it easy to spot and correct a large group of similar errors with a few mouse clicks, minimizing repetitive human interventions. The heat map is aimed at exposing all overlooked errors and helping users progressively approach perfect accuracy in cell lineage reconstruction. Quantitative evaluation proves that eDetect is able to largely improve accuracy within an acceptable time frame, and its performance surpasses the winners of most tasks in the `Cell Tracking Challenge', as measured by biologically relevant metrics

Modeling and Analysis of Subcellular Protein Localization in Hyper-Dimensional Fluorescent Microscopy Images Using Deep Learning Methods

Hyper-dimensional images are informative and become increasingly common in biomedical research. However, the machine learning methods of studying and processing the hyper-dimensional images are underdeveloped. Most of the methods only model the mapping functions between input and output by focusing on the spatial relationship, whereas neglect the temporal and causal relationships. In many cases, the spatial, temporal, and causal relationships are correlated and become a relationship complex. Therefore, only modeling the spatial relationship may result in inaccurate mapping function modeling and lead to undesired output. Despite the importance, there are multiple challenges on modeling the relationship complex, including the model complexity and the data availability. The objective of this dissertation is to comprehensively study the mapping function modeling of the spatial-temporal and the spatial-temporal-causal relationship in hyper-dimensional data with deep learning approaches. The modeling methods are expected to accurately capture the complex relationships in class-level and object-level so that new image processing tools can be developed based on the methods to study the relationships between targets in hyper-dimensional data. In this dissertation, four different cases of relationship complex are studied, including the class-level spatial-temporal-causal relationship and spatial-temporal relationship modeling, and the object-level spatial-temporal-causal relationship and spatial-temporal relationship modeling. The modelings are achieved by deep learning networks that implicitly model the mapping functions with network weight matrix. For spatial-temporal relationship, because the cause factor information is unavailable, discriminative modeling that only relies on available information is studied. For class-level and object-level spatial-temporal-causal relationship, generative modeling is studied with a new deep learning network and three new tools proposed. For spatial-temporal relationship modeling, a state-of-the-art segmentation network has been found to be the best performer over 18 networks. Based on accurate segmentation, we study the object-level temporal dynamics and interactions through dynamics tracking. The multi-object portion tracking (MOPT) method allows object tracking in subcellular level and identifies object events, including object born, dead, split, and fusion. The tracking results is 2.96% higher on consistent tracking accuracy and 35.48% higher on event identification accuracy, compared with the existing state-of-the-art tracking methods. For spatial-temporal-causal relationship modeling, the proposed four-dimensional reslicing generative adversarial network (4DR-GAN) captures the complex relationships between the input and the target proteins. The experimental results on four groups of proteins demonstrate the efficacy of 4DR-GAN compared with the widely used Pix2Pix network. On protein localization prediction (PLP), the predicted localization from 4DR-GAN is more accurate in subcellular localization, temporal consistency, and dynamics. Based on efficient PLP, the digital activation (DA) and digital inactivation (DI) tools allow precise spatial and temporal control on global and local localization manipulation. They allow researchers to study the protein functions and causal relationships by observing the digital manipulation and PLP output response