84,370 research outputs found
A Unified Approach for Learning the Parameters of Sum-Product Networks
We present a unified approach for learning the parameters of Sum-Product
networks (SPNs). We prove that any complete and decomposable SPN is equivalent
to a mixture of trees where each tree corresponds to a product of univariate
distributions. Based on the mixture model perspective, we characterize the
objective function when learning SPNs based on the maximum likelihood
estimation (MLE) principle and show that the optimization problem can be
formulated as a signomial program. We construct two parameter learning
algorithms for SPNs by using sequential monomial approximations (SMA) and the
concave-convex procedure (CCCP), respectively. The two proposed methods
naturally admit multiplicative updates, hence effectively avoiding the
projection operation. With the help of the unified framework, we also show
that, in the case of SPNs, CCCP leads to the same algorithm as Expectation
Maximization (EM) despite the fact that they are different in general.Comment: NIPS 201
Sum-of-Squares Polynomial Flow
Triangular map is a recent construct in probability theory that allows one to
transform any source probability density function to any target density
function. Based on triangular maps, we propose a general framework for
high-dimensional density estimation, by specifying one-dimensional
transformations (equivalently conditional densities) and appropriate
conditioner networks. This framework (a) reveals the commonalities and
differences of existing autoregressive and flow based methods, (b) allows a
unified understanding of the limitations and representation power of these
recent approaches and, (c) motivates us to uncover a new Sum-of-Squares (SOS)
flow that is interpretable, universal, and easy to train. We perform several
synthetic experiments on various density geometries to demonstrate the benefits
(and short-comings) of such transformations. SOS flows achieve competitive
results in simulations and several real-world datasets.Comment: 13 pages, ICML'201
Deep Compression of Sum-Product Networks on Tensor Networks
Sum-product networks (SPNs) represent an emerging class of neural networks
with clear probabilistic semantics and superior inference speed over graphical
models. This work reveals a strikingly intimate connection between SPNs and
tensor networks, thus leading to a highly efficient representation that we call
tensor SPNs (tSPNs). For the first time, through mapping an SPN onto a tSPN and
employing novel optimization techniques, we demonstrate remarkable parameter
compression with negligible loss in accuracy
From Pixels to Buildings: End-to-end Probabilistic Deep Networks for Large-scale Semantic Mapping
We introduce TopoNets, end-to-end probabilistic deep networks for modeling
semantic maps with structure reflecting the topology of large-scale
environments. TopoNets build a unified deep network spanning multiple levels of
abstraction and spatial scales, from pixels representing geometry of local
places to high-level descriptions of semantics of buildings. To this end,
TopoNets leverage complex spatial relations expressed in terms of arbitrary,
dynamic graphs. We demonstrate how TopoNets can be used to perform end-to-end
semantic mapping from partial sensory observations and noisy topological
relations discovered by a robot exploring large-scale office spaces. Thanks to
their probabilistic nature and generative properties, TopoNets extend the
problem of semantic mapping beyond classification. We show that TopoNets
successfully perform uncertain reasoning about yet unexplored space and detect
novel and incongruent environment configurations unknown to the robot. Our
implementation of TopoNets achieves real-time, tractable and exact inference,
which makes these new deep models a promising, practical solution to mobile
robot spatial understanding at scale.Comment: 8 pages, 8 figures, 1 table. The 2019 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS 2019
Data-Dependent Path Normalization in Neural Networks
We propose a unified framework for neural net normalization, regularization
and optimization, which includes Path-SGD and Batch-Normalization and
interpolates between them across two different dimensions. Through this
framework we investigate issue of invariance of the optimization, data
dependence and the connection with natural gradients.Comment: 17 pages, 3 figure
Stochastic And-Or Grammars: A Unified Framework and Logic Perspective
Stochastic And-Or grammars (AOG) extend traditional stochastic grammars of
language to model other types of data such as images and events. In this paper
we propose a representation framework of stochastic AOGs that is agnostic to
the type of the data being modeled and thus unifies various domain-specific
AOGs. Many existing grammar formalisms and probabilistic models in natural
language processing, computer vision, and machine learning can be seen as
special cases of this framework. We also propose a domain-independent inference
algorithm of stochastic context-free AOGs and show its tractability under a
reasonable assumption. Furthermore, we provide two interpretations of
stochastic context-free AOGs as a subset of probabilistic logic, which connects
stochastic AOGs to the field of statistical relational learning and clarifies
their relation with a few existing statistical relational models
Alternating Diffusion Map Based Fusion of Multimodal Brain Connectivity Networks for IQ Prediction
To explain individual differences in development, behavior, and cognition,
most previous studies focused on projecting resting-state functional MRI (fMRI)
based functional connectivity (FC) data into a low-dimensional space via linear
dimensionality reduction techniques, followed by executing analysis operations.
However, linear dimensionality analysis techniques may fail to capture
nonlinearity of brain neuroactivity. Moreover, besides resting-state FC, FC
based on task fMRI can be expected to provide complementary information.
Motivated by these considerations, we nonlinearly fuse resting-state and
task-based FC networks (FCNs) to seek a better representation in this paper. We
propose a framework based on alternating diffusion map (ADM), which extracts
geometry-preserving low-dimensional embeddings that successfully parameterize
the intrinsic variables driving the phenomenon of interest. Specifically, we
first separately build resting-state and task-based FCNs by symmetric positive
definite matrices using sparse inverse covariance estimation for each subject,
and then utilize the ADM to fuse them in order to extract significant
low-dimensional embeddings, which are used as fingerprints to identify
individuals. The proposed framework is validated on the Philadelphia
Neurodevelopmental Cohort data, where we conduct extensive experimental study
on resting-state and fractal -back task fMRI for the classification of
intelligence quotient (IQ). The fusion of resting-state and -back task fMRI
by the proposed framework achieves better classification accuracy than any
single fMRI, and the proposed framework is shown to outperform several other
data fusion methods. To our knowledge, this paper is the first to demonstrate a
successful extension of the ADM to fuse resting-state and task-based fMRI data
for accurate prediction of IQ
Dynamic Few-Shot Visual Learning without Forgetting
The human visual system has the remarkably ability to be able to effortlessly
learn novel concepts from only a few examples. Mimicking the same behavior on
machine learning vision systems is an interesting and very challenging research
problem with many practical advantages on real world vision applications. In
this context, the goal of our work is to devise a few-shot visual learning
system that during test time it will be able to efficiently learn novel
categories from only a few training data while at the same time it will not
forget the initial categories on which it was trained (here called base
categories). To achieve that goal we propose (a) to extend an object
recognition system with an attention based few-shot classification weight
generator, and (b) to redesign the classifier of a ConvNet model as the cosine
similarity function between feature representations and classification weight
vectors. The latter, apart from unifying the recognition of both novel and base
categories, it also leads to feature representations that generalize better on
"unseen" categories. We extensively evaluate our approach on Mini-ImageNet
where we manage to improve the prior state-of-the-art on few-shot recognition
(i.e., we achieve 56.20% and 73.00% on the 1-shot and 5-shot settings
respectively) while at the same time we do not sacrifice any accuracy on the
base categories, which is a characteristic that most prior approaches lack.
Finally, we apply our approach on the recently introduced few-shot benchmark of
Bharath and Girshick [4] where we also achieve state-of-the-art results. The
code and models of our paper will be published on:
https://github.com/gidariss/FewShotWithoutForgettingComment: Accepted at CVPR 2018. Code and models will be published on:
https://github.com/gidariss/FewShotWithoutForgettin
A Connection between Feed-Forward Neural Networks and Probabilistic Graphical Models
Two of the most popular modelling paradigms in computer vision are
feed-forward neural networks (FFNs) and probabilistic graphical models (GMs).
Various connections between the two have been studied in recent works, such as
e.g. expressing mean-field based inference in a GM as an FFN. This paper
establishes a new connection between FFNs and GMs. Our key observation is that
any FFN implements a certain approximation of a corresponding Bayesian network
(BN). We characterize various benefits of having this connection. In
particular, it results in a new learning algorithm for BNs. We validate the
proposed methods for a classification problem on CIFAR-10 dataset and for
binary image segmentation on Weizmann Horse dataset. We show that statistically
learned BNs improve performance, having at the same time essentially better
generalization capability, than their FFN counterparts
Learning Large-Scale Topological Maps Using Sum-Product Networks
In order to perform complex actions in human environments, an autonomous
robot needs the ability to understand the environment, that is, to gather and
maintain spatial knowledge. Topological map is commonly used for representing
large scale, global maps such as floor plans. Although much work has been done
in topological map extraction, we have found little previous work on the
problem of learning the topological map using a probabilistic model. Learning a
topological map means learning the structure of the large-scale space and
dependency between places, for example, how the evidence of a group of places
influence the attributes of other places. This is an important step towards
planning complex actions in the environment. In this thesis, we consider the
problem of using probabilistic deep learning model to learn the topological
map, which is essentially a sparse undirected graph where nodes represent
places annotated with their semantic attributes (e.g. place category). We
propose to use a novel probabilistic deep model, Sum-Product Networks (SPNs),
due to their unique properties. We present two methods for learning topological
maps using SPNs: the place grid method and the template-based method. We
contribute an algorithm that builds SPNs for graphs using template models. Our
experiments evaluate the ability of our models to enable robots to infer
semantic attributes and detect maps with novel semantic attribute arrangements.
Our results demonstrate their understanding of the topological map structure
and spatial relations between places.Comment: 26 pages, 14 figures, senior thesis for departmental honors at the
Allen School of Computer Science and Engineering at the University of
Washingto
- …