A joint classification and segmentation approach for dendritic spine segmentation in 2-photon microscopy images

Shape priors have been successfully used in challenging biomedical imaging problems. However when the shape distribution involves multiple shape classes, leading to a multimodal shape density, effective use of shape priors in segmentation becomes more challenging. In such scenarios, knowing the class of the shape can aid the segmentation process, which is of course unknown a priori. In this paper, we propose a joint classification and segmentation approach for dendritic spine segmentation which infers the class of the spine during segmentation and adapts the remaining segmentation process accordingly. We evaluate our proposed approach on 2-photon microscopy images containing dendritic spines and compare its performance quantitatively to an existing approach based on nonparametric shape priors. Both visual and quantitative results demonstrate the effectiveness of our approach in dendritic spine segmentation

Dendritic Spine Shape Analysis: A Clustering Perspective

Functional properties of neurons are strongly coupled with their morphology. Changes in neuronal activity alter morphological characteristics of dendritic spines. First step towards understanding the structure-function relationship is to group spines into main spine classes reported in the literature. Shape analysis of dendritic spines can help neuroscientists understand the underlying relationships. Due to unavailability of reliable automated tools, this analysis is currently performed manually which is a time-intensive and subjective task. Several studies on spine shape classification have been reported in the literature, however, there is an on-going debate on whether distinct spine shape classes exist or whether spines should be modeled through a continuum of shape variations. Another challenge is the subjectivity and bias that is introduced due to the supervised nature of classification approaches. In this paper, we aim to address these issues by presenting a clustering perspective. In this context, clustering may serve both confirmation of known patterns and discovery of new ones. We perform cluster analysis on two-photon microscopic images of spines using morphological, shape, and appearance based features and gain insights into the spine shape analysis problem. We use histogram of oriented gradients (HOG), disjunctive normal shape models (DNSM), morphological features, and intensity profile based features for cluster analysis. We use x-means to perform cluster analysis that selects the number of clusters automatically using the Bayesian information criterion (BIC). For all features, this analysis produces 4 clusters and we observe the formation of at least one cluster consisting of spines which are difficult to be assigned to a known class. This observation supports the argument of intermediate shape types.Comment: Accepted for BioImageComputing workshop at ECCV 201

Coupled shape priors for dynamic segmentation of dendritic spines

Segmentation of biomedical images is a challenging task, especially when there is low quality or missing data. The use of prior information can provide significant assistance for obtaining more accurate results. In this paper we propose a new approach for dendritic spine segmentation from microscopic images over time, which is motivated by incorporating shape information from previous time points to segment a spine in the current time point. In particular, using a training set consisting of spines in two consecutive time points to construct coupled shape priors, and given the segmentation in the previous time point, we can improve the segmentation process of the spine in the current time point. Our approach has been evaluated on 2-photon microscopy images of dendritic spines and its effectiveness has been demonstrated by both visual and quantitative results

Nonparametric joint shape and feature priors for segmentation of dendritic spines

Multimodal shape density estimation is a challenging task in many biomedical image segmentation problems. Existing techniques in the literature estimate the underlying shape distribution by extending Parzen density estimator to the space of shapes. Such density estimates are only expressed in terms of distances between shapes which may not be sufficient for ensuring accurate segmentation when the observed intensities provide very little information about the object boundaries. In such scenarios, employing additional shape-dependent discriminative features as priors and exploiting both shape and feature priors can aid to the segmentation process. In this paper, we propose a segmentation algorithm that uses nonparametric joint shape and feature priors using Parzen density estimator. The joint prior density estimate is expressed in terms of distances between shapes and distances between features. We incorporate the learned joint shape and feature prior distribution into a maximum a posteriori estimation framework for segmentation. The resulting optimization problem is solved using active contours. We present experimental results on dendritic spine segmentation in 2-photon microscopy images which involve a multimodal shape density

3D dendritic spine segmentation using nonparametric shape priors (3B dendritik dikenlerin parametrik olmayan şekil ön bilgisi kullanılarak bölütlenmesi)

Analyzing morphological and structural changes of dendritic spines in 2-photon microscopy images in time is important for neuroscience researchers. Correct segmentation of dendritic spines is an important step of developing robust and reliable automatic tools for such analysis. In this paper, we propose an approach for segmentation of 3D dendritic spines using nonparametric shape priors. The proposed method learns the prior distribution of shapes through Parzen density estimation on the training set of shapes. Then, the posterior distribution of shapes is obtained by combining the learned prior distribution with a data term in a Bayesian framework. Finally, the segmentation result that maximizes the posterior is found using active contours. Experimental results demonstrate that using nonparametric shape priors leads to better 3D dendritic spine segmentation results

Dendritic spine shape analysis based on two-photon microscopy images

Neuronal morphology and function are highly coupled. In particular, dendritic spine morphology is strongly governed by the incoming neuronal activity. Previously, volumes of dendritic spines have been considered as a primary parameter to study spine morphology and gain insight into structure-function coupling. However, this reductionist approach fails to incorporate the broad spine structure repertoire. First step towards integrating the rich spine morphology information into functional coupling is to classify spine shapes into main spine types suggested in the literature. Due to the lack of reliable automated analysis tools, classification is currently performed manually, which is a time-intensive task and prone to subjectivity. Availability of automated spine shape analysis tools can accelerate this process and help neuroscientists understand underlying structure and function relationship. Several studies on spine shape classification have been reported in the literature, however, there is an on-going debate on whether distinct spine shape classes exist or whether spines should be modeled through a continuum of shape variations. Another challenge is the subjectivity and bias that is introduced due to the supervised nature of classification approaches. This thesis focuses on morphological, shape, and appearance features based methods to perform dendritic spine shape analysis using both clustering and classification approaches. We apply manifold learning methods for dendritic spine classification and observe that ISOMAP implicitly computes prominent features suitable for classification purposes. We also apply linear representation based approach for spine classification and conclude that sparse representation provides slightly better classification performance. We propose 2D and 3D morphological features based approach for spine shape analysis and demonstrate the advantage of 3D morphological features. We also use a deep learning based approach for spine classification and show that mid-level features extracted from Convolutional Neural Networks (CNNs) perform as well as hand-crafted features. We propose a kernel density estimation (KDE) based framework for dendritic spine classification. We evaluate our proposed approaches by comparing labels assigned by a neuroscience expert. Our KDE based framework also enables neuroscientists to analyze separability of spine shape classes in the likelihood ratio space, which leads to further insights about the nature of the spine shape analysis problem. Furthermore, we also propose a methodology for unsupervised learning and clustering of spine shapes. In particular, we use x-means to perform cluster analysis that selects the number of clusters automatically using the Bayesian information criterion (BIC). The objective of clustering in this context is two-fold: confirm the hypothesis of some distinct shape classes and discover new natural groups. We observe that although there are many spines which easily fit into the definition of standard shape types (confirming the hypothesis), there are also a significant number of others which do not comply with standard shape types and demonstrate intermediate properties

An Automated Images-to-Graphs Framework for High Resolution Connectomics

Reconstructing a map of neuronal connectivity is a critical challenge in contemporary neuroscience. Recent advances in high-throughput serial section electron microscopy (EM) have produced massive 3D image volumes of nanoscale brain tissue for the first time. The resolution of EM allows for individual neurons and their synaptic connections to be directly observed. Recovering neuronal networks by manually tracing each neuronal process at this scale is unmanageable, and therefore researchers are developing automated image processing modules. Thus far, state-of-the-art algorithms focus only on the solution to a particular task (e.g., neuron segmentation or synapse identification). In this manuscript we present the first fully automated images-to-graphs pipeline (i.e., a pipeline that begins with an imaged volume of neural tissue and produces a brain graph without any human interaction). To evaluate overall performance and select the best parameters and methods, we also develop a metric to assess the quality of the output graphs. We evaluate a set of algorithms and parameters, searching possible operating points to identify the best available brain graph for our assessment metric. Finally, we deploy a reference end-to-end version of the pipeline on a large, publicly available data set. This provides a baseline result and framework for community analysis and future algorithm development and testing. All code and data derivatives have been made publicly available toward eventually unlocking new biofidelic computational primitives and understanding of neuropathologies.Comment: 13 pages, first two authors contributed equally V2: Added additional experiments and clarifications; added information on infrastructure and pipeline environmen

Generalizable automated pixel-level structural segmentation of medical and biological data

Over the years, the rapid expansion in imaging techniques and equipments has driven the demand for more automation in handling large medical and biological data sets. A wealth of approaches have been suggested as optimal solutions for their respective imaging types. These solutions span various image resolutions, modalities and contrast (staining) mechanisms. Few approaches generalise well across multiple image types, contrasts or resolution. This thesis proposes an automated pixel-level framework that addresses 2D, 2D+t and 3D structural segmentation in a more generalizable manner, yet has enough adaptability to address a number of specific image modalities, spanning retinal funduscopy, sequential fluorescein angiography and two-photon microscopy. The pixel-level segmentation scheme involves: i ) constructing a phase-invariant orientation field of the local spatial neighbourhood; ii ) combining local feature maps with intensity-based measures in a structural patch context; iii ) using a complex supervised learning process to interpret the combination of all the elements in the patch in order to reach a classification decision. This has the advantage of transferability from retinal blood vessels in 2D to neural structures in 3D. To process the temporal components in non-standard 2D+t retinal angiography sequences, we first introduce a co-registration procedure: at the pairwise level, we combine projective RANSAC with a quadratic homography transformation to map the coordinate systems between any two frames. At the joint level, we construct a hierarchical approach in order for each individual frame to be registered to the global reference intra- and inter- sequence(s). We then take a non-training approach that searches in both the spatial neighbourhood of each pixel and the filter output across varying scales to locate and link microvascular centrelines to (sub-) pixel accuracy. In essence, this \link while extract" piece-wise segmentation approach combines the local phase-invariant orientation field information with additional local phase estimates to obtain a soft classification of the centreline (sub-) pixel locations. Unlike retinal segmentation problems where vasculature is the main focus, 3D neural segmentation requires additional exibility, allowing a variety of structures of anatomical importance yet with different geometric properties to be differentiated both from the background and against other structures. Notably, cellular structures, such as Purkinje cells, neural dendrites and interneurons, all display certain elongation along their medial axes, yet each class has a characteristic shape captured by an orientation field that distinguishes it from other structures. To take this into consideration, we introduce a 5D orientation mapping to capture these orientation properties. This mapping is incorporated into the local feature map description prior to a learning machine. Extensive performance evaluations and validation of each of the techniques presented in this thesis is carried out. For retinal fundus images, we compute Receiver Operating Characteristic (ROC) curves on existing public databases (DRIVE & STARE) to assess and compare our algorithms with other benchmark methods. For 2D+t retinal angiography sequences, we compute the error metrics ("Centreline Error") of our scheme with other benchmark methods. For microscopic cortical data stacks, we present segmentation results on both surrogate data with known ground-truth and experimental rat cerebellar cortex two-photon microscopic tissue stacks.Open Acces

Enabling Scalable Neurocartography: Images to Graphs for Discovery

In recent years, advances in technology have enabled researchers to ask new questions predicated on the collection and analysis of big datasets that were previously too large to study. More specifically, many fundamental questions in neuroscience require studying brain tissue at a large scale to discover emergent properties of neural computation, consciousness, and etiologies of brain disorders. A major challenge is to construct larger, more detailed maps (e.g., structural wiring diagrams) of the brain, known as connectomes. Although raw data exist, obstacles remain in both algorithm development and scalable image analysis to enable access to the knowledge within these data volumes. This dissertation develops, combines and tests state-of-the-art algorithms to estimate graphs and glean other knowledge across six orders of magnitude, from millimeter-scale magnetic resonance imaging to nanometer-scale electron microscopy. This work enables scientific discovery across the community and contributes to the tools and services offered by NeuroData and the Open Connectome Project. Contributions include creating, optimizing and evaluating the first known fully-automated brain graphs in electron microscopy data and magnetic resonance imaging data; pioneering approaches to generate knowledge from X-Ray tomography imaging; and identifying and solving a variety of image analysis challenges associated with building graphs suitable for discovery. These methods were applied across diverse datasets to answer questions at scales not previously explored