38 research outputs found
Guided Proofreading of Automatic Segmentations for Connectomics
Automatic cell image segmentation methods in connectomics produce merge and
split errors, which require correction through proofreading. Previous research
has identified the visual search for these errors as the bottleneck in
interactive proofreading. To aid error correction, we develop two classifiers
that automatically recommend candidate merges and splits to the user. These
classifiers use a convolutional neural network (CNN) that has been trained with
errors in automatic segmentations against expert-labeled ground truth. Our
classifiers detect potentially-erroneous regions by considering a large context
region around a segmentation boundary. Corrections can then be performed by a
user with yes/no decisions, which reduces variation of information 7.5x faster
than previous proofreading methods. We also present a fully-automatic mode that
uses a probability threshold to make merge/split decisions. Extensive
experiments using the automatic approach and comparing performance of novice
and expert users demonstrate that our method performs favorably against
state-of-the-art proofreading methods on different connectomics datasets.Comment: Supplemental material available at
http://rhoana.org/guidedproofreading/supplemental.pd
Machine learning of hierarchical clustering to segment 2D and 3D images
We aim to improve segmentation through the use of machine learning tools
during region agglomeration. We propose an active learning approach for
performing hierarchical agglomerative segmentation from superpixels. Our method
combines multiple features at all scales of the agglomerative process, works
for data with an arbitrary number of dimensions, and scales to very large
datasets. We advocate the use of variation of information to measure
segmentation accuracy, particularly in 3D electron microscopy (EM) images of
neural tissue, and using this metric demonstrate an improvement over competing
algorithms in EM and natural images.Comment: 15 pages, 8 figure
Focused Proofreading: Efficiently Extracting Connectomes from Segmented EM Images
Identifying complex neural circuitry from electron microscopic (EM) images
may help unlock the mysteries of the brain. However, identifying this circuitry
requires time-consuming, manual tracing (proofreading) due to the size and
intricacy of these image datasets, thus limiting state-of-the-art analysis to
very small brain regions. Potential avenues to improve scalability include
automatic image segmentation and crowd sourcing, but current efforts have had
limited success. In this paper, we propose a new strategy, focused
proofreading, that works with automatic segmentation and aims to limit
proofreading to the regions of a dataset that are most impactful to the
resulting circuit. We then introduce a novel workflow, which exploits
biological information such as synapses, and apply it to a large dataset in the
fly optic lobe. With our techniques, we achieve significant tracing speedups of
3-5x without sacrificing the quality of the resulting circuit. Furthermore, our
methodology makes the task of proofreading much more accessible and hence
potentially enhances the effectiveness of crowd sourcing
Large-Scale Automatic Reconstruction of Neuronal Processes from Electron Microscopy Images
Automated sample preparation and electron microscopy enables acquisition of
very large image data sets. These technical advances are of special importance
to the field of neuroanatomy, as 3D reconstructions of neuronal processes at
the nm scale can provide new insight into the fine grained structure of the
brain. Segmentation of large-scale electron microscopy data is the main
bottleneck in the analysis of these data sets. In this paper we present a
pipeline that provides state-of-the art reconstruction performance while
scaling to data sets in the GB-TB range. First, we train a random forest
classifier on interactive sparse user annotations. The classifier output is
combined with an anisotropic smoothing prior in a Conditional Random Field
framework to generate multiple segmentation hypotheses per image. These
segmentations are then combined into geometrically consistent 3D objects by
segmentation fusion. We provide qualitative and quantitative evaluation of the
automatic segmentation and demonstrate large-scale 3D reconstructions of
neuronal processes from a volume of brain
tissue over a cube of in each dimension corresponding to
1000 consecutive image sections. We also introduce Mojo, a proofreading tool
including semi-automated correction of merge errors based on sparse user
scribbles
Reinforced Proofreading of Image Segmentation for Connectomics
Department of Computer Science and EngineeringManual connectome reconstruction is a challenging task because of large-scale image data, therefore, an automatic pipelines are needed. Recently, with the usage of deep learning in computer vision, automatic segmentations of electron microscopy (EM) image data are acquired but have the high error rates including merge and split errors, which means it still requires correction through human proofreading. In this thesis, I propose a novel fully automatic proofreading system for 2D segmentation base on reinforcement learning. By mimicking the human proofreading process, the proposed system uses Locator, Merger and Splitter agents for error detection and correction tasks. With an input segmentation image, the Locator agent detects erroneous patches on the input image and then feeds them to Merger and Splitter for correcting split and merge errors respectively. To showcase my system performance, I evaluate it on CREMI data set.ope
Recommended from our members
Augmenting Wiring Diagrams of Neural Circuits with Activity in Larval Drosophila
Neural circuit models explain an animal's behavior as evoked activity of different circuit elements of its nervous system.
Synaptic wiring diagrams mapped from structural imaging of nervous systems guide modeling of neural circuits on the basis of connectivity.
However, connectivity alone may not sufficiently constrain the set of plausible circuit hypotheses for empirical study.
Combining structural imaging of synaptic connectivity with functional information from activity imaging can further constrain these hypotheses to create unequivocal neural circuit models.
This thesis develops computational methods and tools to cross-reference structural and activity imaging of explant larval Drosophila central nervous systems at cellular resolution.
Augmenting synaptic wiring diagrams with activity maps via these methods relates circuit structure and function at the neuronal level on a per-behavior basis.
Neuronal activity of larval central nervous systems expressing pan-neuronal calcium indicators is imaged in a light sheet microscope, which are then structurally imaged with high throughput electron microscopy.
Methods and tools are provided for the assembly of these image volumes, spatial registration between imaging modalities, automated detection of relevant tissue and cellular structures in each, extraction of activity time series, and morphological identification of neurons in structural imaging using reference wiring diagrams mapped from other larvae.
Using these methods, existing wiring diagrams mapped from a reference first instar larva were identified with neurons in a larva augmented with activity information for a neural circuit involved in peristaltic motor control.
This demonstrates the feasibility of the contributed methods to associate the wiring diagrams of arbitrary circuits of interest with activity timeseries across multiple individuals, behaviors, and behavioral bouts.
To demonstrate capability to augment wiring diagrams with information besides activity, these methods are also applied to multiple larvae each expressing specific neurotransmitter labels rather than calcium indicators in the light sheet microscopy.
This work scaffolds future modeling of circuits underlying behavior that can only be mechanistically understood in the context of multi-modal observation of synaptic connectivity, functional activity and molecular markers.
The methods developed also enable comparative connectomics between multiple individuals, which is necessary to study inter-individual variability in circuits and to observe experimental intervention in the development, structure, and function of neural circuits.Howard Hughes Medical Institute Janelia Research Campu
VAST (Volume Annotation and Segmentation Tool): Efficient Manual and Semi-Automatic Labeling of Large 3D Image Stacks
Recent developments in serial-section electron microscopy allow the efficient generation of very large image data sets but analyzing such data poses challenges for software tools. Here we introduce Volume Annotation and Segmentation Tool (VAST), a freely available utility program for generating and editing annotations and segmentations of large volumetric image (voxel) data sets. It provides a simple yet powerful user interface for real-time exploration and analysis of large data sets even in the Petabyte range
NeuTu: Software for Collaborative, Large-Scale, Segmentation-Based Connectome Reconstruction
Reconstructing a connectome from an EM dataset often requires a large effort of proofreading automatically generated segmentations. While many tools exist to enable tracing or proofreading, recent advances in EM imaging and segmentation quality suggest new strategies and pose unique challenges for tool design to accelerate proofreading. Namely, we now have access to very large multi-TB EM datasets where (1) many segments are largely correct, (2) segments can be very large (several GigaVoxels), and where (3) several proofreaders and scientists are expected to collaborate simultaneously. In this paper, we introduce NeuTu as a solution to efficiently proofread large, high-quality segmentation in a collaborative setting. NeuTu is a client program of our high-performance, scalable image database called DVID so that it can easily be scaled up. Besides common features of typical proofreading software, NeuTu tames unprecedentedly large data with its distinguishing functions, including: (1) low-latency 3D visualization of large mutable segmentations; (2) interactive splitting of very large false merges with highly optimized semi-automatic segmentation; (3) intuitive user operations for investigating or marking interesting points in 3D visualization; (4) visualizing proofreading history of a segmentation; and (5) real-time collaborative proofreading with lock-based concurrency control. These unique features have allowed us to manage the workflow of proofreading a large dataset smoothly without dividing them into subsets as in other segmentation-based tools. Most importantly, NeuTu has enabled some of the largest connectome reconstructions as well as interesting discoveries in the fly brain
Topology-Aware Uncertainty for Image Segmentation
Segmentation of curvilinear structures such as vasculature and road networks
is challenging due to relatively weak signals and complex geometry/topology. To
facilitate and accelerate large scale annotation, one has to adopt
semi-automatic approaches such as proofreading by experts. In this work, we
focus on uncertainty estimation for such tasks, so that highly uncertain, and
thus error-prone structures can be identified for human annotators to verify.
Unlike most existing works, which provide pixel-wise uncertainty maps, we
stipulate it is crucial to estimate uncertainty in the units of topological
structures, e.g., small pieces of connections and branches. To achieve this, we
leverage tools from topological data analysis, specifically discrete Morse
theory (DMT), to first capture the structures, and then reason about their
uncertainties. To model the uncertainty, we (1) propose a joint prediction
model that estimates the uncertainty of a structure while taking the
neighboring structures into consideration (inter-structural uncertainty); (2)
propose a novel Probabilistic DMT to model the inherent uncertainty within each
structure (intra-structural uncertainty) by sampling its representations via a
perturb-and-walk scheme. On various 2D and 3D datasets, our method produces
better structure-wise uncertainty maps compared to existing works.Comment: 19 pages, 13 figures, 5 table