Modeling Brain Circuitry over a Wide Range of Scales

If we are ever to unravel the mysteries of brain function at its most fundamental level, we will need a precise understanding of how its component neurons connect to each other. Electron Microscopes (EM) can now provide the nanometer resolution that is needed to image synapses, and therefore connections, while Light Microscopes (LM) see at the micrometer resolution required to model the 3D structure of the dendritic network. Since both the topology and the connection strength are integral parts of the brain's wiring diagram, being able to combine these two modalities is critically important. In fact, these microscopes now routinely produce high-resolution imagery in such large quantities that the bottleneck becomes automated processing and interpretation, which is needed for such data to be exploited to its full potential. In this paper, we briefly review the Computer Vision techniques we have developed at EPFL to address this need. They include delineating dendritic arbors from LM imagery, segmenting organelles from EM, and combining the two into a consistent representation

Simultaneous Segmentation and Anatomical Labeling of the Cerebral Vasculature

We present a novel algorithm for the simultaneous segmentation and anatomical labeling of the cerebral vasculature. The method first constructs an over-complete graph capturing the vasculature. It then selects and labels the subset of edges that most likely represents the true vasculature. Unlike existing approaches that first attempt to obtain a good segmentation and then perform labeling, we jointly optimize for both by simultaneously taking into account the image evidence and the prior knowledge about the geometry and connectivity of the vasculature. This results in an Integer Program (IP), which we solve optimally using a branch-and-cut algorithm. We evaluate our approach on a public dataset of 50 cerebral MRA images, and demonstrate that it compares favorably against state-of-the-art methods

Extracting Vascular Networks under Physiological Constraints via Integer Programming

Abstract. We introduce an integer programming-based approach to vessel net-work extraction that enforces global physiological constraints on the vessel struc-ture and learn this prior from a high-resolution reference network. The method accounts for both image evidence and geometric relationships between vessels by formulating and solving an integer programming problem. Starting from an over-connected network, it is pruning vessel stumps and spurious connections by evaluating bifurcation angle and connectivity of the graph. We utilize a high-resolution micro computed tomography (µCT) dataset of a cerebrovascular corro-sion cast to obtain a reference network, perform experiments on micro magnetic resonance angiography (µMRA) images of mouse brains and discuss properties of the networks obtained under different tracking and pruning approaches.

Semi-Automated Reconstruction of Curvilinear Structures in Noisy 2D images and 3D image stacks

We propose a new approach to semi-automated delineation of curvilinear structures in a wide range of imaging modalities. Earlier approaches lack robustness to imaging noise, do not provide radius estimates for the structures and operate only on single channel images. In contrast, ours makes use of the color information, when available, and generates accurate centreline location and radius estimates with minimal supervision. We demonstrate this on a wide range of datasets ranging from a 2D dataset of aerial images to 3D micrographs of neurites

Inference of Curvilinear Structure based on Learning a Ranking Function and Graph Theory

To detect curvilinear structures in natural images, we propose a novel rankinglearning system and an abstract curvilinear shape inference algorithm based on graph theory. Weanalyze the curvilinear structures as a set of small line segments. In this work, the rankings ofthe line segments are exploited to systematize the topological feature of the curvilinear structures.Structured Support Vector Machine is employed to learn the ranking function that predicts thecorrespondence of the given line segments and the latent curvilinear structures. We first extractcurvilinear features using morphological profiles and steerable filtering responses. Also, we proposean orientation-aware feature descriptor and a feature grouping operator to improve the structuralintegrity during the learning process. To infer the curvilinear structure, we build a graph based onthe output rankings of the line segments. We progressively reconstruct the curvilinear structureby looking for paths between remote vertices in the graph. Experimental results show that theproposed algorithm faithfully detects the curvilinear structures within various datasets

Automated Neuron Reconstruction from 3D Fluorescence Microscopy Images Using Sequential Monte Carlo Estimation

Microscopic images of neuronal cells provide essential structural information about the key constituents of the brain and form the basis of many neuroscientific studies. Computational analyses of the morphological properties of the captured neurons require first converting the structural information into digital tree-like reconstructions. Many dedicated computational methods and corresponding software tools have been and are continuously being developed with the aim to automate this step while achieving human-comparable reconstruction accuracy. This pursuit is hampered by the immense diversity and intricacy of neuronal morphologies as well as the often low quality and ambiguity of the images. Here we present a novel method we developed in an effort to improve the robustness of digital reconstruction against these complicating factors. The method is based on probabilistic filtering by sequential Monte Carlo estimation and uses prediction and update models designed specifically for tracing neuronal branches in microscopic image stacks. Moreover, it uses multiple probabilistic traces to arrive at a more robust, ensemble reconstruction. The proposed method was evaluated on fluorescence microscopy image stacks of single neurons and dense neuronal networks with expert manual annotations serving as the gold standard, as well as on synthetic images with known ground truth. The results indicate that our method performs well under varying experimental conditions and compares favorably to state-of-the-art alternative methods

Automatic Multi-Model Fitting for Blood Vessel Extraction

Blood vessel extraction and visualization in 2D images or 3D volumes is an essential clinical task. A blood vessel system is an example of a tubular tree like structure, and fully automated reconstruction of tubular tree like structures remains an open computer vision problem. Most vessel extraction methods are based on the vesselness measure. A vesselness measure, usually based on the eigenvalues of the Hessian matrix, assigns a high value to a voxel that is likely to be a part of a blood vessel. After the vesselness measure is computed, most methods extract vessels based on the shortest paths connecting voxels with a high measure of vesselness. Our approach is quite different. We also start with the vesselness measure, but instead of computing shortest paths, we propose to fit a geometric of vessel system to the vesselness measure. Fitting a geometric model has the advantage that we can choose a model with desired properties and the appropriate goodness-of-fit function to control the fitting results. Changing the model and goodness-of-fit function allows us to change the properties of the reconstructed vessel system structure in a principled way. In contrast, with shortest paths, any undesirable reconstruction properties, such as short-cutting, is addressed by developing ad-hock procedures that are not easy to control. Since the geometric model has to be fitted to a discrete set of points, we threshold the vesselness measure to extract voxels that are likely to be vessels, and fit our geometric model to these thresholded voxels. Our geometric model is a piecewise-line segment model. That is we approximate the vessel structure as a collection of 3D straight line segments of various lengths and widths. This can be regarded as the problem of fitting multiple line segments, that is a multi-model fitting problem. We approach the multi-model fitting problem in the global energy optimization framework. That is we formulate a global energy function that reflects the goodness of fit of our piecewise line segment model to the thresholded vesselness voxels and we use the efficient and effective graph cut algorithm to optimize the energy. Our global energy function consists of the data, smoothness and label cost. The data cost encourages a good geometric fit of each voxel to the line segment it is being assigned to. The smoothness cost encourages nearby line segments to have similar angles, thus encouraging smoother blood vessels. The label cost penalizes overly complex models, that is, it encourages to explain the data with fewer line segment models. We apply our algorithm to the challenging 3D data that are micro-CT images of a mouse heart and obtain promising results

Reconstructing Loopy Curvilinear Structures Using Integer Programming

We propose a novel approach to automated delineation of linear structures that form complex and potentially loopy networks. This is in contrast to earlier approaches that usually assume a tree topology for the networks. At the heart of our method is an Integer Programming formulation that allows us to find the global optimum of an objective function designed to allow cycles but penalize spurious junctions and early terminations. We demonstrate that it outperforms state-of-the-art techniques on a wide range of datasets. (Aerial) (Confocal