1,405 research outputs found
Neuron Segmentation Using Deep Complete Bipartite Networks
In this paper, we consider the problem of automatically segmenting neuronal
cells in dual-color confocal microscopy images. This problem is a key task in
various quantitative analysis applications in neuroscience, such as tracing
cell genesis in Danio rerio (zebrafish) brains. Deep learning, especially using
fully convolutional networks (FCN), has profoundly changed segmentation
research in biomedical imaging. We face two major challenges in this problem.
First, neuronal cells may form dense clusters, making it difficult to correctly
identify all individual cells (even to human experts). Consequently,
segmentation results of the known FCN-type models are not accurate enough.
Second, pixel-wise ground truth is difficult to obtain. Only a limited amount
of approximate instance-wise annotation can be collected, which makes the
training of FCN models quite cumbersome. We propose a new FCN-type deep
learning model, called deep complete bipartite networks (CB-Net), and a new
scheme for leveraging approximate instance-wise annotation to train our
pixel-wise prediction model. Evaluated using seven real datasets, our proposed
new CB-Net model outperforms the state-of-the-art FCN models and produces
neuron segmentation results of remarkable qualityComment: miccai 201
Improving Sequence-to-Sequence Learning via Optimal Transport
Sequence-to-sequence models are commonly trained via maximum likelihood
estimation (MLE). However, standard MLE training considers a word-level
objective, predicting the next word given the previous ground-truth partial
sentence. This procedure focuses on modeling local syntactic patterns, and may
fail to capture long-range semantic structure. We present a novel solution to
alleviate these issues. Our approach imposes global sequence-level guidance via
new supervision based on optimal transport, enabling the overall
characterization and preservation of semantic features. We further show that
this method can be understood as a Wasserstein gradient flow trying to match
our model to the ground truth sequence distribution. Extensive experiments are
conducted to validate the utility of the proposed approach, showing consistent
improvements over a wide variety of NLP tasks, including machine translation,
abstractive text summarization, and image captioning
Associative content-addressable networks with exponentially many robust stable states
The brain must robustly store a large number of memories, corresponding to
the many events encountered over a lifetime. However, the number of memory
states in existing neural network models either grows weakly with network size
or recall fails catastrophically with vanishingly little noise. We construct an
associative content-addressable memory with exponentially many stable states
and robust error-correction. The network possesses expander graph connectivity
on a restricted Boltzmann machine architecture. The expansion property allows
simple neural network dynamics to perform at par with modern error-correcting
codes. Appropriate networks can be constructed with sparse random connections,
glomerular nodes, and associative learning using low dynamic-range weights.
Thus, sparse quasi-random structures---characteristic of important
error-correcting codes---may provide for high-performance computation in
artificial neural networks and the brain.Comment: 42 pages, 8 figure
Undirected Graphical Models as Approximate Posteriors
The representation of the approximate posterior is a critical aspect of
effective variational autoencoders (VAEs). Poor choices for the approximate
posterior have a detrimental impact on the generative performance of VAEs due
to the mismatch with the true posterior. We extend the class of posterior
models that may be learned by using undirected graphical models. We develop an
efficient method to train undirected approximate posteriors by showing that the
gradient of the training objective with respect to the parameters of the
undirected posterior can be computed by backpropagation through Markov chain
Monte Carlo updates. We apply these gradient estimators for training discrete
VAEs with Boltzmann machines as approximate posteriors and demonstrate that
undirected models outperform previous results obtained using directed graphical
models. Our implementation is available at https://github.com/QuadrantAI/dvaess .Comment: Accepted to ICML 202
Convolutional Bipartite Attractor Networks
In human perception and cognition, a fundamental operation that brains
perform is interpretation: constructing coherent neural states from noisy,
incomplete, and intrinsically ambiguous evidence. The problem of interpretation
is well matched to an early and often overlooked architecture, the attractor
network---a recurrent neural net that performs constraint satisfaction,
imputation of missing features, and clean up of noisy data via energy
minimization dynamics. We revisit attractor nets in light of modern deep
learning methods and propose a convolutional bipartite architecture with a
novel training loss, activation function, and connectivity constraints. We
tackle larger problems than have been previously explored with attractor nets
and demonstrate their potential for image completion and super-resolution. We
argue that this architecture is better motivated than ever-deeper feedforward
models and is a viable alternative to more costly sampling-based generative
methods on a range of supervised and unsupervised tasks
Anomaly Detection and Correction in Large Labeled Bipartite Graphs
Binary classification problems can be naturally modeled as bipartite graphs,
where we attempt to classify right nodes based on their left adjacencies. We
consider the case of labeled bipartite graphs in which some labels and edges
are not trustworthy. Our goal is to reduce noise by identifying and fixing
these labels and edges.
We first propose a geometric technique for generating random graph instances
with untrustworthy labels and analyze the resulting graph properties. We focus
on generating graphs which reflect real-world data, where degree and label
frequencies follow power law distributions.
We review several algorithms for the problem of detection and correction,
proposing novel extensions and making observations specific to the bipartite
case. These algorithms range from math programming algorithms to discrete
combinatorial algorithms to Bayesian approximation algorithms to machine
learning algorithms.
We compare the performance of all these algorithms using several metrics and,
based on our observations, identify the relative strengths and weaknesses of
each individual algorithm.Comment: 36 pages, 4 figure
Speed/accuracy trade-offs for modern convolutional object detectors
The goal of this paper is to serve as a guide for selecting a detection
architecture that achieves the right speed/memory/accuracy balance for a given
application and platform. To this end, we investigate various ways to trade
accuracy for speed and memory usage in modern convolutional object detection
systems. A number of successful systems have been proposed in recent years, but
apples-to-apples comparisons are difficult due to different base feature
extractors (e.g., VGG, Residual Networks), different default image resolutions,
as well as different hardware and software platforms. We present a unified
implementation of the Faster R-CNN [Ren et al., 2015], R-FCN [Dai et al., 2016]
and SSD [Liu et al., 2015] systems, which we view as "meta-architectures" and
trace out the speed/accuracy trade-off curve created by using alternative
feature extractors and varying other critical parameters such as image size
within each of these meta-architectures. On one extreme end of this spectrum
where speed and memory are critical, we present a detector that achieves real
time speeds and can be deployed on a mobile device. On the opposite end in
which accuracy is critical, we present a detector that achieves
state-of-the-art performance measured on the COCO detection task.Comment: Accepted to CVPR 201
Deep Neural Networks
Deep Neural Networks (DNNs) are universal function approximators providing
state-of- the-art solutions on wide range of applications. Common perceptual
tasks such as speech recognition, image classification, and object tracking are
now commonly tackled via DNNs. Some fundamental problems remain: (1) the lack
of a mathematical framework providing an explicit and interpretable
input-output formula for any topology, (2) quantification of DNNs stability
regarding adversarial examples (i.e. modified inputs fooling DNN predictions
whilst undetectable to humans), (3) absence of generalization guarantees and
controllable behaviors for ambiguous patterns, (4) leverage unlabeled data to
apply DNNs to domains where expert labeling is scarce as in the medical field.
Answering those points would provide theoretical perspectives for further
developments based on a common ground. Furthermore, DNNs are now deployed in
tremendous societal applications, pushing the need to fill this theoretical gap
to ensure control, reliability, and interpretability.Comment: Technical Repor
Probabilistic Semantic Retrieval for Surveillance Videos with Activity Graphs
We present a novel framework for finding complex activities matching
user-described queries in cluttered surveillance videos. The wide diversity of
queries coupled with unavailability of annotated activity data limits our
ability to train activity models. To bridge the semantic gap we propose to let
users describe an activity as a semantic graph with object attributes and
inter-object relationships associated with nodes and edges, respectively. We
learn node/edge-level visual predictors during training and, at test-time,
propose to retrieve activity by identifying likely locations that match the
semantic graph. We formulate a novel CRF based probabilistic activity
localization objective that accounts for mis-detections, mis-classifications
and track-losses, and outputs a likelihood score for a candidate grounded
location of the query in the video. We seek groundings that maximize overall
precision and recall. To handle the combinatorial search over all
high-probability groundings, we propose a highest precision subgraph matching
algorithm. Our method outperforms existing retrieval methods on benchmarked
datasets.Comment: 1520-9210 (c) 2018 IEEE. This paper has been accepted by IEEE
Transactions on Multimedia. Print ISSN: 1520-9210. Online ISSN: 1941-0077.
Preprint link is https://ieeexplore.ieee.org/document/8438958
On Regret-Optimal Learning in Decentralized Multi-player Multi-armed Bandits
We consider the problem of learning in single-player and multiplayer
multiarmed bandit models. Bandit problems are classes of online learning
problems that capture exploration versus exploitation tradeoffs. In a
multiarmed bandit model, players can pick among many arms, and each play of an
arm generates an i.i.d. reward from an unknown distribution. The objective is
to design a policy that maximizes the expected reward over a time horizon for a
single player setting and the sum of expected rewards for the multiplayer
setting. In the multiplayer setting, arms may give different rewards to
different players. There is no separate channel for coordination among the
players. Any attempt at communication is costly and adds to regret. We propose
two decentralizable policies, (-) and -, that can be used in both single player and multiplayer settings.
These policies are shown to yield expected regret that grows at most as
O(). It is well known that is the lower bound on
the rate of growth of regret even in a centralized case. The proposed
algorithms improve on prior work where regret grew at O(). More
fundamentally, these policies address the question of additional cost incurred
in decentralized online learning, suggesting that there is at most an
-factor cost in terms of order of regret. This solves a problem of
relevance in many domains and had been open for a while
- …