Error Resilient Video Coding Using Bitstream Syntax And Iterative Microscopy Image Segmentation

There has been a dramatic increase in the amount of video traffic over the Internet in past several years. For applications like real-time video streaming and video conferencing, retransmission of lost packets is often not permitted. Popular video coding standards such as H.26x and VPx make use of spatial-temporal correlations for compression, typically making compressed bitstreams vulnerable to errors. We propose several adaptive spatial-temporal error concealment approaches for subsampling-based multiple description video coding. These adaptive methods are based on motion and mode information extracted from the H.26x video bitstreams. We also present an error resilience method using data duplication in VPx video bitstreams. A recent challenge in image processing is the analysis of biomedical images acquired using optical microscopy. Due to the size and complexity of the images, automated segmentation methods are required to obtain quantitative, objective and reproducible measurements of biological entities. In this thesis, we present two techniques for microscopy image analysis. Our first method, “Jelly Filling” is intended to provide 3D segmentation of biological images that contain incompleteness in dye labeling. Intuitively, this method is based on filling disjoint regions of an image with jelly-like fluids to iteratively refine segments that represent separable biological entities. Our second method selectively uses a shape-based function optimization approach and a 2D marked point process simulation, to quantify nuclei by their locations and sizes. Experimental results exhibit that our proposed methods are effective in addressing the aforementioned challenges

Segmentation of fluorescence microscopy images using three dimensional active contours with inhomogeneity correction

Image segmentation is an important step in the quantitative analysis of fluorescence microscopy data. Since fluorescence microscopy volumes suffer from intensity inhomogeneity, low image contrast and limited depth resolution, poor edge details, and irregular structure shape, segmentation still remains a challenging problem. This paper describes a nuclei segmentation method for fluorescence microscopy based on the use of three dimensional (3D) active contours with inhomogeneity correction. The correction information utilizes 3D volume information while addressing intensity inhomogeneity across vertical and horizontal directions. Experimental results demonstrate that the proposed method achieves better performance than other reported methods

Nuclei Segmentation of Fluorescence Microscopy Images Using Three Dimensional Convolutional Neural Networks

Fluorescence microscopy enables one to visualize subcellular structures of living tissue or cells in three dimensions. This is especially true for two-photon microscopy using near-infrared light which can image deeper into tissue. To characterize and analyze biological structures, nuclei segmentation is a prerequisite step. Due to the complexity and size of the image data sets, manual segmentation is prohibitive. This paper describes a fully 3D nuclei segmentation method using three dimensional convolutional neural networks. To train the network, synthetic volumes with corresponding labeled volumes are automatically generated. Our results from multiple data sets demonstrate that our method can successfully segment nuclei in 3D

Automatic Hotspots Detection for Intracellular Calcium Analysis in Fluorescence Microscopic Videos

In recent years, life-cell imaging techniques and their software applications have become powerful tools to investigate complex biological mechanisms such as calcium signalling. In this paper, we propose an automated framework to detect areas inside cells that show changes in their calcium concentration i.e. the regions of interests or hotspots, based on videos taken after loading living mouse cardiomyocytes with fluorescent calcium reporter dyes. The proposed system allows an objective and efficient analysis through the following four key stages: (1) Pre-processing to enhance video quality, (2) First level segmentation to detect candidate hotspots based on adaptive thresholding on the frame level, (3) Second-level segmentation to fuse and identify the best hotspots from the entire video by proposing the concept of calcium fluorescence hit-ratio, and (4) Extraction of the changes of calcium fluorescence over time per hotspot. From the extracted signals, different measurements are calculated such as maximum peak amplitude, area under the curve, peak frequency, and inter-spike interval of calcium changes. The system was tested using calcium imaging data collected from Heart muscle cells. The paper argues that the automated proposal offers biologists a tool to speed up the processing time and mitigate the consequences of inter-intra observer variability

Segmentation of biological images containing multitarget labeling using the jelly filling framework

Biomedical imaging when combined with digital image analysis is capable of quantitative morphological and physiological characterizations of biological structures. Recent fluorescence microscopy techniques can collect hundreds of focal plane images from deeper tissue volumes, thus enabling characterization of three-dimensional (3-D) biological structures at subcellular resolution. Automatic analysis methods are required to obtain quantitative, objective, and reproducible measurements of biological quantities. However, these images typically contain many artifacts such as poor edge details, nonuniform brightness, and distortions that vary along different axes, all of which complicate the automatic image analysis. Another challenge is due to "multitarget labeling," in which a single probe labels multiple biological entities in acquired images. We present a "jelly filling" method for segmentation of 3-D biological images containing multitarget labeling. Intuitively, our iterative segmentation method is based on filling disjoint tubule regions of an image with a jelly-like fluid. This helps in the detection of components that are "floating" within a labeled jelly. Experimental results show that our proposed method is effective in segmenting important biological quantities

Investigating the 3D chromatin architecture with fluorescence microscopy

Chromatin is an assembly of DNA and nuclear proteins, which on the one hand has the function to properly store the 2 meters of DNA of a diploid human nucleus in a small volume and on the other hand regulates the accessibility of specific DNA segments for proteins. Many cellular processes like gene expression and DNA repair are affected by the three-dimensional architecture of chromatin. Cohesin is an important and well-studied protein that affects three-dimensional chromatin organization. One of the functions of this motor protein is the active generation of specific domain structures (topologically associating domains (TADs)) by the process of loop extrusion. Studies of cohesin depleted cells showed that TAD structures were lost on a population average. Due to this finding, the question arose, to what extent the functional nuclear architecture, that can be detected by confocal and structured illumination microscopy, is impaired when cells were cohesin depleted. The work presented in this thesis could show that the structuring of the nucleus in areas with different chromatin densities including the localization of important nuclear proteins as well as replication patterns was retained. Interestingly, cohesin depleted cells proceeded through an endomitosis leading to the formation of multilobulated nuclei. Obviously, important structural features of chromatin can form even in the absence of cohesin. In the here presented work, fluorescence microscopic methods were used throughout, and an innovative technique was developed, that allows flexible labeling of proteins with different fluorophores in fixed cells. With this technique DNA as well as peptide nucleic acid (PNA) oligonucleotides can be site-specifically coupled to antibodies via the Tub-tag technology and visualized by complementary fluorescently labeled oligonucleotides. The advantages and disadvantages of PNAs as docking strands are discussed in this thesis as well as the use of PNAs in fluorescence in situ hybridization (FISH). In the next study, which is part of this work, a combination of FISH and super-resolution microscopy was used. There it could be shown that DNA segments of 5 kb can form both compact and elongated configurations in regulatory active as well as inactive chromatin. Coarse-grained modeling of these microscopic data, in agreement with published data from other groups, has suggested that elongated configurations occur more frequently in DNA segments in which the occupancy of nucleosomes is reduced. The microscopically measured distance distributions could only be simulated with models that assume different densities of nucleosomes in the population. Another result of this study was that inactive chromatin - as expected - shows a high level of compaction, which can hardly be explained with common coarse-grained models. It is possible that environmental effects that are difficult to simulate play a role here. Chromatin is a highly dynamic structure, and its architecture is constantly changing, be it through active processes such as the effect of cohesin investigated here or through thermodynamic interactions of nucleosomes as they are simulated in coarse-grained models. It will take a long time until we adequately understand these dynamic processes and their interplay

Visualizing calcium flux in freely moving nematode embryos

Author Posting. © The Author(s), 2017. This is the author's version of the work. It is posted here by permission of Cell Press for personal use, not for redistribution. The definitive version was published in Biophysical Journal 112 (2017): 1975-1983, doi:10.1016/j.bpj.2017.02.035.The lack of physiological recordings from Caenorhabditis elegans embryos stands in stark contrast to the comprehensive anatomical and gene expression datasets already available. Using light-sheet fluorescence microscopy (LSFM) to address the challenges associated with functional imaging at this developmental stage, we recorded calcium dynamics in muscles and neurons and developed analysis strategies to relate activity and movement. In muscles, we found that the initiation of twitching was associated with a spreading calcium wave in a dorsal muscle bundle. Correlated activity in muscle bundles was linked with early twitching and eventual coordinated movement. To identify neuronal correlates of behavior, we monitored brain-wide activity with subcellular resolution and identified a particularly active cell associated with muscle contractions. Finally, imaging neurons of a well-defined adult motor circuit, we found that reversals in the eggshell correlated with calcium transients in AVA interneurons.E.A. and A.K. acknowledge support from the Grass Fellowship Program and D. C-R. and H.S. acknowledge the Whitman Fellowship program at MBL. This work was supported by the intramural research program of the National Institute of Biomedical Imaging and Bioengineering and NIH grants U01 HD075602 and R24OD016474 to D.C-R and A.K.2018-05-0

Image analysis and statistical modeling for applications in cytometry and bioprocess control

Today, signal processing has a central role in many of the advancements in systems biology. Modern signal processing is required to provide efficient computational solutions to unravel complex problems that are either arduous or impossible to obtain using conventional approaches. For example, imaging-based high-throughput experiments enable cells to be examined at even subcellular level yielding huge amount of image data. Cytometry is an integral part of such experiments and involves measurement of different cell parameters which requires extraction of quantitative experimental values from cell microscopy images. In order to do that for such large number of images, fast and accurate automated image analysis methods are required. In another example, modeling of bioprocesses and their scale-up is a challenging task where different scales have different parameters and often there are more variables than the available number of observations thus requiring special methodology. In many biomedical cell microscopy studies, it is necessary to analyze the images at single cell or even subcellular level since owing to the heterogeneity of cell populations the population-averaged measurements are often inconclusive. Moreover, the emergence of imaging-based high-content screening experiments, especially for drug design, has put single cell analysis at the forefront since it is required to study the dynamics of single-cell gene expressions for tracking and quantification of cell phenotypic variations. The ability to perform single cell analysis depends on the accuracy of image segmentation in detecting individual cells from images. However, clumping of cells at both nuclei and cytoplasm level hinders accurate cell image segmentation. Part of this thesis work concentrates on developing accurate automated methods for segmentation of bright field as well as multichannel fluorescence microscopy images of cells with an emphasis on clump splitting so that cells are separated from each other as well as from background. The complexity in bioprocess development and control crave for the usage of computational modeling and data analysis approaches for process optimization and scale-up. This is also asserted by the fact that obtaining a priori knowledge needed for the development of traditional scale-up criteria may at times be difficult. Moreover, employment of efficient process modeling may provide the added advantage of automatic identification of influential control parameters. Determination of the values of the identified parameters and the ability to predict them at different scales help in process control and in achieving their scale-up. Bioprocess modeling and control can also benefit from single cell analysis where the latter could add a new dimension to the former once imaging-based in-line sensors allow for monitoring of key variables governing the processes. In this thesis we exploited signal processing techniques for statistical modeling of bioprocess and its scale-up as well as for development of fully automated methods for biomedical cell microscopy image segmentation beginning from image pre-processing and initial segmentation to clump splitting and image post-processing with the goal to facilitate the high-throughput analysis. In order to highlight the contribution of this work, we present three application case studies where we applied the developed methods to solve the problems of cell image segmentation and bioprocess modeling and scale-up

Deep learning for digitized histology image analysis

“Cervical cancer is the fourth most frequent cancer that affects women worldwide. Assessment of cervical intraepithelial neoplasia (CIN) through histopathology remains as the standard for absolute determination of cancer. The examination of tissue samples under a microscope requires considerable time and effort from expert pathologists. There is a need to design an automated tool to assist pathologists for digitized histology slide analysis. Pre-cervical cancer is generally determined by examining the CIN which is the growth of atypical cells from the basement membrane (bottom) to the top of the epithelium. It has four grades, including: Normal, CIN1, CIN2, and CIN3. In this research, different facets of an automated digitized histology epithelium assessment pipeline have been explored to mimic the pathologist diagnostic approach. The entire pipeline from slide to epithelium CIN grade has been designed and developed using deep learning models and imaging techniques to analyze the whole slide image (WSI). The process is as follows: 1) identification of epithelium by filtering the regions extracted from a low-resolution image with a binary classifier network; 2) epithelium segmentation; 3) deep regression for pixel-wise segmentation of epithelium by patch-based image analysis; 4) attention-based CIN classification with localized sequential feature modeling. Deep learning-based nuclei detection by superpixels was performed as an extension of our research. Results from this research indicate an improved performance of CIN assessment over state-of-the-art methods for nuclei segmentation, epithelium segmentation, and CIN classification, as well as the development of a prototype WSI-level tool”--Abstract, page iv

Combinatorial Optimization Algorithms for Hypergraph Matching with Application to Posture Identification in Embryonic Caenorhabditis elegans

Point-set matching defines the task in computer vision of identifying a one-to-one alignment between two sets of points. Existing techniques rely on simple relationships between point-sets in order to efficiently find optimal correspondences between larger sets. Modern methodology precludes application to more challenging point-set matching tasks which benefit from interdependent modeling. This thesis addresses a gap in combinatorial optimization literature by enhancing leading methods in both graph matching and multiple object tracking (MOT) to more flexibly use data-driven point-set matching models. Presented contributions are inspired by Caenorhabditis elegans, a transparent free-living roundworm frequently studied in developmental biology and neurobiology. The C. elegans embryo, containing around 550 cells at hatch, can be used for cell tracking studies to understand how cell movement drives the development of specific embryonic tissues and organ functions. The development of muscle cells complicates analyses during late-stage development, as embryos begin twitching due to muscular activity. The sporadic twitches cause cells to move violently and unpredictably, invalidating traditional cell tracking approaches. The embryo possesses seam cells, a set of 20 cells which together act as fiducial markers to approximate the coiled embryo's body. Novel optimization algorithms and data-driven hypergraphical models leveraging the correlated movement among seam cells are used to forward research on C. elegans embryogenesis. We contribute two optimization algorithms applicable in differing conditions to interdependent point-set matching. The first algorithm, Exact Hypergraph Matching (EHGM), exactly solves the n-adic assignment problem by casting the problem as hypergraph matching. The algorithm obtains solutions to highly interdependent seam cell identification models. The second optimization algorithm, Multiple Hypothesis Hypergraph Tracking (MHHT), adapts traditional multiple hypothesis tracking with hypergraphical data association. Results from both studies highlight improved performance over established methods while providing adaptable optimization tools for multiple academic communities. The canonical point-set matching task is solved efficiently under strict assumptions of frame-to-frame transformations. Challenging situations arising from non-rigid displacements between frames will complicate established methods. Particularly, limitations in fluorescence microscopy paired with muscular twitching in late-stage embryonic C. elegans yield adversarial point-set matching tasks. Seam cell identification is cast as an assignment problem; detected cells in images are uniquely identified through a combinatorial optimization algorithm. Existing graph matching methods are underequipped to contextualize the coiled embryonic position in sparsely imaged samples. Both the lack of an effective point-set matching model and an efficient algorithm for solving the resulting optimization problem limit computationally driven solutions to identify seam cells in acquired image volumes. We cast the n-adic problem as hypergraph matching and present an exact algorithm to solve the resulting optimization problem. EHGM adapts the branch-and-bound paradigm to dynamically identify a globally optimal correspondence; it is the first algorithm capable of solving the underlying optimization problem. Our algorithm and accompanying data-driven hypergraphical models identify seam cells more accurately than established point-set matching methods. The final hours of embryogenesis encompass the moments in which C. elegans assumes motor control and begins exhibiting behavior. Rapid imaging of the seam cells provides insight into the embryo’s movement as a proxy for behavior. However, seam cell tracking is especially challenging due to both muscular twitching and the low dose required to gently image the embryo without perturbing development. Current methods in MOT rely on independent object trajectories undergoing smooth motion to effectively track large numbers of objects. Multiple Hypothesis Tracking (MHT) is the foremost method for challenging MOT tasks, yet the method cannot model correlated object movements. We contribute Multiple Hypothesis Hypergraph Tracking (MHHT) as an extension of MHT, which performs interdependent object tracking by jointly representing objects as a hypergraph. We apply MHHT to track seam cell nuclei during late-stage embryogenesis. Data-driven hypergraphical models more accurately track seam cells than traditional MHT based approaches. Analysis of time-lapse embryonic postures and behavioral motifs reveal a stereotyped developmental progression in C. elegans. Further analysis uncovers late-stage motility defects in unc-13 mutants