15,373 research outputs found
Neural Network Operations and Susuki-Trotter evolution of Neural Network States
It was recently proposed to leverage the representational power of artificial
neural networks, in particular Restricted Boltzmann Machines, in order to model
complex quantum states of many-body systems [Science, 355(6325), 2017]. States
represented in this way, called Neural Network States (NNSs), were shown to
display interesting properties like the ability to efficiently capture
long-range quantum correlations. However, identifying an optimal neural network
representation of a given state might be challenging, and so far this problem
has been addressed with stochastic optimization techniques. In this work we
explore a different direction. We study how the action of elementary quantum
operations modifies NNSs. We parametrize a family of many body quantum
operations that can be directly applied to states represented by Unrestricted
Boltzmann Machines, by just adding hidden nodes and updating the network
parameters. We show that this parametrization contains a set of universal
quantum gates, from which it follows that the state prepared by any quantum
circuit can be expressed as a Neural Network State with a number of hidden
nodes that grows linearly with the number of elementary operations in the
circuit. This is a powerful representation theorem (which was recently obtained
with different methods) but that is not directly useful, since there is no
general and efficient way to extract information from this unrestricted
description of quantum states. To circumvent this problem, we propose a
step-wise procedure based on the projection of Unrestricted quantum states to
Restricted quantum states. In turn, two approximate methods to perform this
projection are discussed. In this way, we show that it is in principle possible
to approximately optimize or evolve Neural Network States without relying on
stochastic methods such as Variational Monte Carlo, which are computationally
expensive
A Theory of Cheap Control in Embodied Systems
We present a framework for designing cheap control architectures for embodied
agents. Our derivation is guided by the classical problem of universal
approximation, whereby we explore the possibility of exploiting the agent's
embodiment for a new and more efficient universal approximation of behaviors
generated by sensorimotor control. This embodied universal approximation is
compared with the classical non-embodied universal approximation. To exemplify
our approach, we present a detailed quantitative case study for policy models
defined in terms of conditional restricted Boltzmann machines. In contrast to
non-embodied universal approximation, which requires an exponential number of
parameters, in the embodied setting we are able to generate all possible
behaviors with a drastically smaller model, thus obtaining cheap universal
approximation. We test and corroborate the theory experimentally with a
six-legged walking machine. The experiments show that the sufficient controller
complexity predicted by our theory is tight, which means that the theory has
direct practical implications. Keywords: cheap design, embodiment, sensorimotor
loop, universal approximation, conditional restricted Boltzmann machineComment: 27 pages, 10 figure
Neural Networks retrieving Boolean patterns in a sea of Gaussian ones
Restricted Boltzmann Machines are key tools in Machine Learning and are
described by the energy function of bipartite spin-glasses. From a statistical
mechanical perspective, they share the same Gibbs measure of Hopfield networks
for associative memory. In this equivalence, weights in the former play as
patterns in the latter. As Boltzmann machines usually require real weights to
be trained with gradient descent like methods, while Hopfield networks
typically store binary patterns to be able to retrieve, the investigation of a
mixed Hebbian network, equipped with both real (e.g., Gaussian) and discrete
(e.g., Boolean) patterns naturally arises. We prove that, in the challenging
regime of a high storage of real patterns, where retrieval is forbidden, an
extra load of Boolean patterns can still be retrieved, as long as the ratio
among the overall load and the network size does not exceed a critical
threshold, that turns out to be the same of the standard
Amit-Gutfreund-Sompolinsky theory. Assuming replica symmetry, we study the case
of a low load of Boolean patterns combining the stochastic stability and
Hamilton-Jacobi interpolating techniques. The result can be extended to the
high load by a non rigorous but standard replica computation argument.Comment: 16 pages, 1 figur
Training Restricted Boltzmann Machines on Word Observations
The restricted Boltzmann machine (RBM) is a flexible tool for modeling
complex data, however there have been significant computational difficulties in
using RBMs to model high-dimensional multinomial observations. In natural
language processing applications, words are naturally modeled by K-ary discrete
distributions, where K is determined by the vocabulary size and can easily be
in the hundreds of thousands. The conventional approach to training RBMs on
word observations is limited because it requires sampling the states of K-way
softmax visible units during block Gibbs updates, an operation that takes time
linear in K. In this work, we address this issue by employing a more general
class of Markov chain Monte Carlo operators on the visible units, yielding
updates with computational complexity independent of K. We demonstrate the
success of our approach by training RBMs on hundreds of millions of word
n-grams using larger vocabularies than previously feasible and using the
learned features to improve performance on chunking and sentiment
classification tasks, achieving state-of-the-art results on the latter
- …