39 research outputs found
Modeling Brain Circuitry over a Wide Range of Scales
If we are ever to unravel the mysteries of brain function at its most
fundamental level, we will need a precise understanding of how its component
neurons connect to each other. Electron Microscopes (EM) can now provide the
nanometer resolution that is needed to image synapses, and therefore
connections, while Light Microscopes (LM) see at the micrometer resolution
required to model the 3D structure of the dendritic network. Since both the
topology and the connection strength are integral parts of the brain's wiring
diagram, being able to combine these two modalities is critically important.
In fact, these microscopes now routinely produce high-resolution imagery in
such large quantities that the bottleneck becomes automated processing and
interpretation, which is needed for such data to be exploited to its full
potential. In this paper, we briefly review the Computer Vision techniques we
have developed at EPFL to address this need. They include delineating dendritic
arbors from LM imagery, segmenting organelles from EM, and combining the two
into a consistent representation
Detection of curved lines with B-COSFIRE filters: A case study on crack delineation
The detection of curvilinear structures is an important step for various
computer vision applications, ranging from medical image analysis for
segmentation of blood vessels, to remote sensing for the identification of
roads and rivers, and to biometrics and robotics, among others. %The visual
system of the brain has remarkable abilities to detect curvilinear structures
in noisy images. This is a nontrivial task especially for the detection of thin
or incomplete curvilinear structures surrounded with noise. We propose a
general purpose curvilinear structure detector that uses the brain-inspired
trainable B-COSFIRE filters. It consists of four main steps, namely nonlinear
filtering with B-COSFIRE, thinning with non-maximum suppression, hysteresis
thresholding and morphological closing. We demonstrate its effectiveness on a
data set of noisy images with cracked pavements, where we achieve
state-of-the-art results (F-measure=0.865). The proposed method can be employed
in any computer vision methodology that requires the delineation of curvilinear
and elongated structures.Comment: Accepted at Computer Analysis of Images and Patterns (CAIP) 201
Seeing Tree Structure from Vibration
Humans recognize object structure from both their appearance and motion;
often, motion helps to resolve ambiguities in object structure that arise when
we observe object appearance only. There are particular scenarios, however,
where neither appearance nor spatial-temporal motion signals are informative:
occluding twigs may look connected and have almost identical movements, though
they belong to different, possibly disconnected branches. We propose to tackle
this problem through spectrum analysis of motion signals, because vibrations of
disconnected branches, though visually similar, often have distinctive natural
frequencies. We propose a novel formulation of tree structure based on a
physics-based link model, and validate its effectiveness by theoretical
analysis, numerical simulation, and empirical experiments. With this
formulation, we use nonparametric Bayesian inference to reconstruct tree
structure from both spectral vibration signals and appearance cues. Our model
performs well in recognizing hierarchical tree structure from real-world videos
of trees and vessels.Comment: ECCV 2018. The first two authors contributed equally to this work.
Project page: http://tree.csail.mit.edu
Enforcing connectivity of 3D linear structures using their 2D projections
Many biological and medical tasks require the delineation of 3D curvilinear
structures such as blood vessels and neurites from image volumes. This is
typically done using neural networks trained by minimizing voxel-wise loss
functions that do not capture the topological properties of these structures.
As a result, the connectivity of the recovered structures is often wrong, which
lessens their usefulness. In this paper, we propose to improve the 3D
connectivity of our results by minimizing a sum of topology-aware losses on
their 2D projections. This suffices to increase the accuracy and to reduce the
annotation effort required to provide the required annotated training data
Graph Refinement based Airway Extraction using Mean-Field Networks and Graph Neural Networks
Graph refinement, or the task of obtaining subgraphs of interest from
over-complete graphs, can have many varied applications. In this work, we
extract trees or collection of sub-trees from image data by, first deriving a
graph-based representation of the volumetric data and then, posing the tree
extraction as a graph refinement task. We present two methods to perform graph
refinement. First, we use mean-field approximation (MFA) to approximate the
posterior density over the subgraphs from which the optimal subgraph of
interest can be estimated. Mean field networks (MFNs) are used for inference
based on the interpretation that iterations of MFA can be seen as feed-forward
operations in a neural network. This allows us to learn the model parameters
using gradient descent. Second, we present a supervised learning approach using
graph neural networks (GNNs) which can be seen as generalisations of MFNs.
Subgraphs are obtained by training a GNN-based graph refinement model to
directly predict edge probabilities. We discuss connections between the two
classes of methods and compare them for the task of extracting airways from 3D,
low-dose, chest CT data. We show that both the MFN and GNN models show
significant improvement when compared to one baseline method, that is similar
to a top performing method in the EXACT'09 Challenge, and a 3D U-Net based
airway segmentation model, in detecting more branches with fewer false
positives.Comment: Accepted for publication at Medical Image Analysis. 14 page
Examining the Measures of Street Connectivity in the American City and their Interdependencies as applied in Practice
This work expands much of the preceding research on the measures of existing street networks and provides a foundation for continued study of urban morphology in the American city. It provides a definition for categorizing street networks based on their morphological characteristics, and it uses the categories to demonstrate inconsistencies in the interdependencies of the measures of street connectivity. In summary, it argues that the simple measures of the elements in a street network, those of road segment length and block area, are not as powerful a descriptor of street connectivity as some of the more complex or composite measures. It charges that if our intent is to increase connectivity and the directness of routes within a street network (Handy, Paterson, & Butler, 2003), then we need to use a measure, such as composite connectivity, to describe configuration
Nilpotent Approximations of Sub-Riemannian Distances for Fast Perceptual Grouping of Blood Vessels in 2D and 3D
We propose an efficient approach for the grouping of local orientations
(points on vessels) via nilpotent approximations of sub-Riemannian distances in
the 2D and 3D roto-translation groups and . In our distance
approximations we consider homogeneous norms on nilpotent groups that locally
approximate , and which are obtained via the exponential and logarithmic
map on . In a qualitative validation we show that the norms provide
accurate approximations of the true sub-Riemannian distances, and we discuss
their relations to the fundamental solution of the sub-Laplacian on .
The quantitative experiments further confirm the accuracy of the
approximations. Quantitative results are obtained by evaluating perceptual
grouping performance of retinal blood vessels in 2D images and curves in
challenging 3D synthetic volumes. The results show that 1) sub-Riemannian
geometry is essential in achieving top performance and 2) that grouping via the
fast analytic approximations performs almost equally, or better, than
data-adaptive fast marching approaches on and .Comment: 18 pages, 9 figures, 3 tables, in review at JMI
Curvilinear Structure Enhancement in Biomedical Images
Curvilinear structures can appear in many different areas and at a variety of scales. They can be axons and dendrites in the brain, blood vessels in the fundus, streets, rivers or fractures in buildings, and others. So, it is essential to study curvilinear structures in many fields such as neuroscience, biology, and cartography regarding image processing.
Image processing is an important field for the help to aid in biomedical imaging especially the diagnosing the disease. Image enhancement is the early step of image analysis.
In this thesis, I focus on the research, development, implementation, and validation of 2D and 3D curvilinear structure enhancement methods, recently established. The proposed methods are based on phase congruency, mathematical morphology, and tensor representation concepts.
First, I have introduced a 3D contrast independent phase congruency-based enhancement approach. The obtained results demonstrate the proposed approach is robust against the contrast variations in 3D biomedical images.
Second, I have proposed a new mathematical morphology-based approach called the bowler-hat transform. In this approach, I have combined the mathematical morphology with a local tensor representation of curvilinear structures in images.
The bowler-hat transform is shown to give better results than comparison methods on challenging data such as retinal/fundus images. The bowler-hat transform is shown to give better results than comparison methods on challenging data such as retinal/fundus images. Especially the proposed method is quite successful while enhancing of curvilinear structures at junctions.
Finally, I have extended the bowler-hat approach to the 3D version to prove the applicability, reliability, and ability of it in 3D