Learning image segmentation and hierarchies by learning ultrametric distances

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences, 2009.Cataloged from PDF version of thesis.Includes bibliographical references (p. 100-105).In this thesis I present new contributions to the fields of neuroscience and computer science. The neuroscientific contribution is a new technique for automatically reconstructing complete neural networks from densely stained 3d electron micrographs of brain tissue. The computer science contribution is a new machine learning method for image segmentation and the development of a new theory for supervised hierarchy learning based on ultrametric distance functions. It is well-known that the connectivity of neural networks in the brain can have a dramatic influence on their computational function . However, our understanding of the complete connectivity of neural circuits has been quite impoverished due to our inability to image all the connections between all the neurons in biological network. Connectomics is an emerging field in neuroscience that aims to revolutionize our understanding of the function of neural circuits by imaging and reconstructing entire neural circuits. In this thesis, I present an automated method for reconstructing neural circuitry from 3d electron micrographs of brain tissue. The cortical column, a basic unit of cortical microcircuitry, will produce a single 3d electron micrograph measuring many 100s terabytes once imaged and contain neurites from well over 100,000 different neurons. It is estimated that tracing the neurites in such a volume by hand would take several thousand human years. Automated circuit tracing methods are thus crucial to the success of connectomics. In computer vision, the circuit reconstruction problem of tracing neurites is known as image segmentation. Segmentation is a grouping problem where image pixels belonging to the same neurite are clustered together. While many algorithms for image segmentation exist, few have parameters that can be optimized using groundtruth data to extract maximum performance on a specialized dataset. In this thesis, I present the first machine learning method to directly minimize an image segmentation error. It is based the theory of ultrametric distances and hierarchical clustering. Image segmentation is posed as the problem of learning and classifying ultrametric distances between image pixels. Ultrametric distances on point set have the special property that(cont.) they correspond exactly to hierarchical clustering of the set. This special property implies hierarchical clustering can be learned by directly learning ultrametric distances. In this thesis, I develop convolutional networks as a machine learning architecture for image processing. I use this powerful pattern recognition architecture with many tens of thousands of free parameters for predicting affinity graphs and detecting object boundaries in images. When trained using ultrametric learning, the convolutional network based algorithm yields an extremely efficient linear-time segmentation algorithm. In this thesis, I develop methods for assessing the quality of image segmentations produced by manual human efforts or by automated computer algorithms. These methods are crucial for comparing the performance of different segmentation methods and is used through out the thesis to demonstrate the quality of the reconstructions generated by the methods in this thesis.by Srinivas C. Turaga.Ph.D

Large Scale Image Segmentation with Structured Loss based Deep Learning for Connectome Reconstruction

We present a method combining affinity prediction with region agglomeration, which improves significantly upon the state of the art of neuron segmentation from electron microscopy (EM) in accuracy and scalability. Our method consists of a 3D U-NET, trained to predict affinities between voxels, followed by iterative region agglomeration. We train using a structured loss based on MALIS, encouraging topologically correct segmentations obtained from affinity thresholding. Our extension consists of two parts: First, we present a quasi-linear method to compute the loss gradient, improving over the original quadratic algorithm. Second, we compute the gradient in two separate passes to avoid spurious gradient contributions in early training stages. Our predictions are accurate enough that simple learning-free percentile-based agglomeration outperforms more involved methods used earlier on inferior predictions. We present results on three diverse EM datasets, achieving relative improvements over previous results of 27%, 15%, and 250%. Our findings suggest that a single method can be applied to both nearly isotropic block-face EM data and anisotropic serial sectioned EM data. The runtime of our method scales linearly with the size of the volume and achieves a throughput of about 2.6 seconds per megavoxel, qualifying our method for the processing of very large datasets

Programmable 3D snapshot microscopy with Fourier convolutional networks

3D snapshot microscopy enables fast volumetric imaging by capturing a 3D volume in a single 2D camera image, and has found a variety of biological applications such as whole brain imaging of fast neural activity in larval zebrafish. The optimal microscope design for this optical 3D-to-2D encoding is both sample- and task-dependent, with no general solution known. Highly programmable optical elements create new possibilities for sample-specific computational optimization of microscope parameters, e.g. tuning the collection of light for a given sample structure. We perform such optimization with deep learning, using a differentiable wave-optics simulation of light propagation through a programmable microscope and a neural network to reconstruct volumes from the microscope image. We introduce a class of global kernel Fourier convolutional neural networks which can efficiently decode information from multiple depths in the volume, globally encoded across a 3D snapshot image. We show that our proposed networks succeed in large field of view volume reconstruction and microscope parameter optimization where traditional networks fail. We also show that our networks outperform the state-of-the-art learned reconstruction algorithms for lensless computational photography.Comment: Make zebrafish Types A,B,C,D more clea

Crowdsourcing the creation of image segmentation algorithms for connectomics

To stimulate progress in automating the reconstruction of neural circuits, we organized the first international challenge on 2D segmentation of electron microscopic (EM) images of the brain. Participants submitted boundary maps predicted for a test set of images, and were scored based on their agreement with a consensus of human expert annotations. The winning team had no prior experience with EM images, and employed a convolutional network. This “deep learning” approach has since become accepted as a standard for segmentation of EM images. The challenge has continued to accept submissions, and the best so far has resulted from cooperation between two teams. The challenge has probably saturated, as algorithms cannot progress beyond limits set by ambiguities inherent in 2D scoring and the size of the test dataset. Retrospective evaluation of the challenge scoring system reveals that it was not sufficiently robust to variations in the widths of neurite borders. We propose a solution to this problem, which should be useful for a future 3D segmentation challenge

Community-based benchmarking improves spike rate inference from two-photon calcium imaging data

In recent years, two-photon calcium imaging has become a standard tool to probe the function of neural circuits and to study computations in neuronal populations. However, the acquired signal is only an indirect measurement of neural activity due to the comparatively slow dynamics of fluorescent calcium indicators. Different algorithms for estimating spike rates from noisy calcium measurements have been proposed in the past, but it is an open question how far performance can be improved. Here, we report the results of the spikefinder challenge, launched to catalyze the development of new spike rate inference algorithms through crowd-sourcing. We present ten of the submitted algorithms which show improved performance compared to previously evaluated methods. Interestingly, the top-performing algorithms are based on a wide range of principles from deep neural networks to generative models, yet provide highly correlated estimates of the neural activity. The competition shows that benchmark challenges can drive algorithmic developments in neuroscience