Multi-stage Multi-recursive-input Fully Convolutional Networks for Neuronal Boundary Detection

In the field of connectomics, neuroscientists seek to identify cortical connectivity comprehensively. Neuronal boundary detection from the Electron Microscopy (EM) images is often done to assist the automatic reconstruction of neuronal circuit. But the segmentation of EM images is a challenging problem, as it requires the detector to be able to detect both filament-like thin and blob-like thick membrane, while suppressing the ambiguous intracellular structure. In this paper, we propose multi-stage multi-recursive-input fully convolutional networks to address this problem. The multiple recursive inputs for one stage, i.e., the multiple side outputs with different receptive field sizes learned from the lower stage, provide multi-scale contextual boundary information for the consecutive learning. This design is biologically-plausible, as it likes a human visual system to compare different possible segmentation solutions to address the ambiguous boundary issue. Our multi-stage networks are trained end-to-end. It achieves promising results on two public available EM segmentation datasets, the mouse piriform cortex dataset and the ISBI 2012 EM dataset.Comment: Accepted by ICCV201

Recursive Training of 2D-3D Convolutional Networks for Neuronal Boundary Detection

Efforts to automate the reconstruction of neural circuits from 3D electron microscopic (EM) brain images are critical for the field of connectomics. An important computation for reconstruction is the detection of neuronal boundaries. Images acquired by serial section EM, a leading 3D EM technique, are highly anisotropic, with inferior quality along the third dimension. For such images, the 2D max-pooling convolutional network has set the standard for performance at boundary detection. Here we achieve a substantial gain in accuracy through three innovations. Following the trend towards deeper networks for object recognition, we use a much deeper network than previously employed for boundary detection. Second, we incorporate 3D as well as 2D filters, to enable computations that use 3D context. Finally, we adopt a recursively trained architecture in which a first network generates a preliminary boundary map that is provided as input along with the original image to a second network that generates a final boundary map. Backpropagation training is accelerated by ZNN, a new implementation of 3D convolutional networks that uses multicore CPU parallelism for speed. Our hybrid 2D-3D architecture could be more generally applicable to other types of anisotropic 3D images, including video, and our recursive framework for any image labeling problem

Doctor of Philosophy

dissertationNeuroscientists are developing new imaging techniques and generating large volumes of data in an effort to understand the complex structure of the nervous system. The complexity and size of this data makes human interpretation a labor intensive task. To aid in the analysis, new segmentation techniques for identifying neurons in these feature rich datasets are required. However, the extremely anisotropic resolution of the data makes segmentation and tracking across slices difficult. Furthermore, the thickness of the slices can make the membranes of the neurons hard to identify. Similarly, structures can change significantly from one section to the next due to slice thickness which makes tracking difficult. This thesis presents a complete method for segmenting many neurons at once in two-dimensional (2D) electron microscopy images and reconstructing and visualizing them in three-dimensions (3D). First, we present an advanced method for identifying neuron membranes in 2D, necessary for whole neuron segmentation, using a machine learning approach. The method described uses a series of artificial neural networks (ANNs) in a framework combined with a feature vector that is composed of image and context; intensities sampled over a stencil neighborhood. Several ANNs are applied in series allowing each ANN to use the classification context; provided by the previous network to improve detection accuracy. To improve the membrane detection, we use information from a nonlinear alignment of sequential learned membrane images in a final ANN that improves membrane detection in each section. The final output, the detected membranes, are used to obtain 2D segmentations of all the neurons in an image. We also present a method that constructs 3D neuron representations by formulating the problem of finding paths through sets of sections as an optimal path computation, which applies a cost function to the identification of a cell from one section to the next and solves this optimization problem using Dijkstras algorithm. This basic formulation accounts for variability or inconsistencies between sections and prioritizes cells based on the evidence of their connectivity. Finally, we present a tool that combines these techniques with a visual user interface that enables users to quickly segment whole neurons in large volumes

Doctor of Philosophy

dissertationElectron microscopy can visualize synapses at nanometer resolution, and can thereby capture the fine structure of these contacts. However, this imaging method lacks three key elements: temporal information, protein visualization, and large volume reconstruction. For my dissertation, I developed three methods in electron microscopy that overcame these limitations. First, I developed a method to freeze neurons at any desired time point after a stimulus to study synaptic vesicle cycle. Second, I developed a method to couple super-resolution fluorescence microscopy and electron microscopy to pinpoint the location of proteins in electron micrographs at nanometer resolution. Third, I collaborated with computer scientists to develop methods for semi-automated reconstruction of nervous system. I applied these techniques to answer two fundamental questions in synaptic biology. Which vesicles fuse in response to a stimulus? How are synaptic vesicles recovered at synapses after fusion? Only vesicles that are in direct contact with plasma membrane fuse upon stimulation. The active zone in C. elegans is broad, but primed vesicles are concentrated around the dense projection. Following exocytosis of synaptic vesicles, synaptic vesicle membrane was recovered rapidly at two distinct locations at a synapse: the dense projection and adherens junctions. These studies suggest that there may be a novel form of ultrafast endocytosis

A Pipeline for Volume Electron Microscopy of the Caenorhabditis elegans Nervous System.

The "connectome," a comprehensive wiring diagram of synaptic connectivity, is achieved through volume electron microscopy (vEM) analysis of an entire nervous system and all associated non-neuronal tissues. White et al. (1986) pioneered the fully manual reconstruction of a connectome using Caenorhabditis elegans. Recent advances in vEM allow mapping new C. elegans connectomes with increased throughput, and reduced subjectivity. Current vEM studies aim to not only fill the remaining gaps in the original connectome, but also address fundamental questions including how the connectome changes during development, the nature of individuality, sexual dimorphism, and how genetic and environmental factors regulate connectivity. Here we describe our current vEM pipeline and projected improvements for the study of the C. elegans nervous system and beyond