765,182 research outputs found
Learning from Minimum Entropy Queries in a Large Committee Machine
In supervised learning, the redundancy contained in random examples can be
avoided by learning from queries. Using statistical mechanics, we study
learning from minimum entropy queries in a large tree-committee machine. The
generalization error decreases exponentially with the number of training
examples, providing a significant improvement over the algebraic decay for
random examples. The connection between entropy and generalization error in
multi-layer networks is discussed, and a computationally cheap algorithm for
constructing queries is suggested and analysed.Comment: 4 pages, REVTeX, multicol, epsf, two postscript figures. To appear in
Physical Review E (Rapid Communications
Phase Transitions of Neural Networks
The cooperative behaviour of interacting neurons and synapses is studied
using models and methods from statistical physics. The competition between
training error and entropy may lead to discontinuous properties of the neural
network. This is demonstrated for a few examples: Perceptron, associative
memory, learning from examples, generalization, multilayer networks, structure
recognition, Bayesian estimate, on-line training, noise estimation and time
series generation.Comment: Plenary talk for MINERVA workshop on mesoscopics, fractals and neural
networks, Eilat, March 1997 Postscript Fil
Geometric deep learning: going beyond Euclidean data
Many scientific fields study data with an underlying structure that is a
non-Euclidean space. Some examples include social networks in computational
social sciences, sensor networks in communications, functional networks in
brain imaging, regulatory networks in genetics, and meshed surfaces in computer
graphics. In many applications, such geometric data are large and complex (in
the case of social networks, on the scale of billions), and are natural targets
for machine learning techniques. In particular, we would like to use deep
neural networks, which have recently proven to be powerful tools for a broad
range of problems from computer vision, natural language processing, and audio
analysis. However, these tools have been most successful on data with an
underlying Euclidean or grid-like structure, and in cases where the invariances
of these structures are built into networks used to model them. Geometric deep
learning is an umbrella term for emerging techniques attempting to generalize
(structured) deep neural models to non-Euclidean domains such as graphs and
manifolds. The purpose of this paper is to overview different examples of
geometric deep learning problems and present available solutions, key
difficulties, applications, and future research directions in this nascent
field
Strategic Learning and the Topology of Social Networks
We consider a group of strategic agents who must each repeatedly take one of
two possible actions. They learn which of the two actions is preferable from
initial private signals, and by observing the actions of their neighbors in a
social network.
We show that the question of whether or not the agents learn efficiently
depends on the topology of the social network. In particular, we identify a
geometric "egalitarianism" condition on the social network that guarantees
learning in infinite networks, or learning with high probability in large
finite networks, in any equilibrium. We also give examples of non-egalitarian
networks with equilibria in which learning fails.Comment: 30 pages, one figur
- …