Recursive Training of 2D-3D Convolutional Networks for Neuronal Boundary Detection

Efforts to automate the reconstruction of neural circuits from 3D electron microscopic (EM) brain images are critical for the field of connectomics. An important computation for reconstruction is the detection of neuronal boundaries. Images acquired by serial section EM, a leading 3D EM technique, are highly anisotropic, with inferior quality along the third dimension. For such images, the 2D max-pooling convolutional network has set the standard for performance at boundary detection. Here we achieve a substantial gain in accuracy through three innovations. Following the trend towards deeper networks for object recognition, we use a much deeper network than previously employed for boundary detection. Second, we incorporate 3D as well as 2D filters, to enable computations that use 3D context. Finally, we adopt a recursively trained architecture in which a first network generates a preliminary boundary map that is provided as input along with the original image to a second network that generates a final boundary map. Backpropagation training is accelerated by ZNN, a new implementation of 3D convolutional networks that uses multicore CPU parallelism for speed. Our hybrid 2D-3D architecture could be more generally applicable to other types of anisotropic 3D images, including video, and our recursive framework for any image labeling problem

Comparative Connectomics.

We introduce comparative connectomics, the quantitative study of cross-species commonalities and variations in brain network topology that aims to discover general principles of network architecture of nervous systems and the identification of species-specific features of brain connectivity. By comparing connectomes derived from simple to more advanced species, we identify two conserved themes of wiring: the tendency to organize network topology into communities that serve specialized functionality and the general drive to enable high topological integration by means of investment of neural resources in short communication paths, hubs, and rich clubs. Within the space of wiring possibilities that conform to these common principles, we argue that differences in connectome organization between closely related species support adaptations in cognition and behavior.We thank Lianne Scholtens, Jim Rilling, Tom Schoenemann for discussions and comments. MPvdH was supported by a VENI (# 451-12-001) grant from the Netherlands Organization for Scientific Research (NWO) and a Fellowship of MQ.This is the author accepted manuscript. The final version is available from Elsevier via https://doi.org/10.1016/j.tics.2016.03.00

Building connectomes using diffusion MRI: why, how and but

Why has diffusion MRI become a principal modality for mapping connectomes in vivo? How do different image acquisition parameters, fiber tracking algorithms and other methodological choices affect connectome estimation? What are the main factors that dictate the success and failure of connectome reconstruction? These are some of the key questions that we aim to address in this review. We provide an overview of the key methods that can be used to estimate the nodes and edges of macroscale connectomes, and we discuss open problems and inherent limitations. We argue that diffusion MRI-based connectome mapping methods are still in their infancy and caution against blind application of deep white matter tractography due to the challenges inherent to connectome reconstruction. We review a number of studies that provide evidence of useful microstructural and network properties that can be extracted in various independent and biologically-relevant contexts. Finally, we highlight some of the key deficiencies of current macroscale connectome mapping methodologies and motivate future developments

Machine Learning for Instance Segmentation

Volumetric Electron Microscopy images can be used for connectomics, the study of brain connectivity at the cellular level. A prerequisite for this inquiry is the automatic identification of neural cells, which requires machine learning algorithms and in particular efficient image segmentation algorithms. In this thesis, we develop new algorithms for this task. In the first part we provide, for the first time in this field, a method for training a neural network to predict optimal input data for a watershed algorithm. We demonstrate its superior performance compared to other segmentation methods of its category. In the second part, we develop an efficient watershed-based algorithm for weighted graph partitioning, the \emph{Mutex Watershed}, which uses negative edge-weights for the first time. We show that it is intimately related to the multicut and has a cutting edge performance on a connectomics challenge. Our algorithm is currently used by the leaders of two connectomics challenges. Finally, motivated by inpainting neural networks, we create a method to learn the graph weights without any supervision

Estimating functional brain networks by incorporating a modularity prior

Functional brain network analysis has become one principled way of revealing informative organization architectures in healthy brains, and providing sensitive biomarkers for diagnosis of neurological disorders. Prior to any post hoc analysis, however, a natural issue is how to construct “ideal” brain networks given, for example, a set of functional magnetic resonance imaging (fMRI) time series associated with different brain regions. Although many methods have been developed, it is currently still an open field to estimate biologically meaningful and statistically robust brain networks due to our limited understanding of the human brain as well as complex noises in the observed data. Motivated by the fact that the brain is organized with modular structures, in this paper, we propose a novel functional brain network modeling scheme by encoding a modularity prior under a matrix-regularized network learning framework, and further formulate it as a sparse low-rank graph learning problem, which can be solved by an efficient optimization algorithm. Then, we apply the learned brain networks to identify patients with mild cognitive impairment (MCI) from normal controls. We achieved 89.01% classification accuracy even with a simple feature selection and classification pipeline, which significantly outperforms the conventional brain network construction methods. Moreover, we further explore brain network features that contributed to MCI identification, and discovered potential biomarkers for personalized diagnosis

Enabling Scalable Neurocartography: Images to Graphs for Discovery

In recent years, advances in technology have enabled researchers to ask new questions predicated on the collection and analysis of big datasets that were previously too large to study. More specifically, many fundamental questions in neuroscience require studying brain tissue at a large scale to discover emergent properties of neural computation, consciousness, and etiologies of brain disorders. A major challenge is to construct larger, more detailed maps (e.g., structural wiring diagrams) of the brain, known as connectomes. Although raw data exist, obstacles remain in both algorithm development and scalable image analysis to enable access to the knowledge within these data volumes. This dissertation develops, combines and tests state-of-the-art algorithms to estimate graphs and glean other knowledge across six orders of magnitude, from millimeter-scale magnetic resonance imaging to nanometer-scale electron microscopy. This work enables scientific discovery across the community and contributes to the tools and services offered by NeuroData and the Open Connectome Project. Contributions include creating, optimizing and evaluating the first known fully-automated brain graphs in electron microscopy data and magnetic resonance imaging data; pioneering approaches to generate knowledge from X-Ray tomography imaging; and identifying and solving a variety of image analysis challenges associated with building graphs suitable for discovery. These methods were applied across diverse datasets to answer questions at scales not previously explored