851 research outputs found
Beyond the storage capacity: data driven satisfiability transition
Data structure has a dramatic impact on the properties of neural networks,
yet its significance in the established theoretical frameworks is poorly
understood. Here we compute the Vapnik-Chervonenkis entropy of a kernel machine
operating on data grouped into equally labelled subsets. At variance with the
unstructured scenario, entropy is non-monotonic in the size of the training
set, and displays an additional critical point besides the storage capacity.
Remarkably, the same behavior occurs in margin classifiers even with randomly
labelled data, as is elucidated by identifying the synaptic volume encoding the
transition. These findings reveal aspects of expressivity lying beyond the
condensed description provided by the storage capacity, and they indicate the
path towards more realistic bounds for the generalization error of neural
networks.Comment: 5 pages, 2 figure
Replica Symmetry Breaking in Cold Atoms and Spin Glasses
We consider a system composed by N atoms trapped within a multimode cavity,
whose theoretical description is captured by a disordered multimode Dicke
model. We show that in the resonant, zero field limit the system exactly
realizes the Sherrington-Kirkpatrick model. Upon a redefinition of the
temperature, the same dynamics is realized in the dispersive, strong field
limit. This regime also gives access to spin-glass observables which can be
used to detect Replica Symmetry Breaking.Comment: 6 pages, 3 figure
Counting the learnable functions of structured data
Cover's function counting theorem is a milestone in the theory of artificial
neural networks. It provides an answer to the fundamental question of
determining how many binary assignments (dichotomies) of points in
dimensions can be linearly realized. Regrettably, it has proved hard to extend
the same approach to more advanced problems than the classification of points.
In particular, an emerging necessity is to find methods to deal with structured
data, and specifically with non-pointlike patterns. A prominent case is that of
invariant recognition, whereby identification of a stimulus is insensitive to
irrelevant transformations on the inputs (such as rotations or changes in
perspective in an image). An object is therefore represented by an extended
perceptual manifold, consisting of inputs that are classified similarly. Here,
we develop a function counting theory for structured data of this kind, by
extending Cover's combinatorial technique, and we derive analytical expressions
for the average number of dichotomies of generically correlated sets of
patterns. As an application, we obtain a closed formula for the capacity of a
binary classifier trained to distinguish general polytopes of any dimension.
These results may help extend our theoretical understanding of generalization,
feature extraction, and invariant object recognition by neural networks
Dicke simulators with emergent collective quantum computational abilities
Using an approach inspired from Spin Glasses, we show that the multimode
disordered Dicke model is equivalent to a quantum Hopfield network. We propose
variational ground states for the system at zero temperature, which we
conjecture to be exact in the thermodynamic limit. These ground states contain
the information on the disordered qubit-photon couplings. These results lead to
two intriguing physical implications. First, once the qubit-photon couplings
can be engineered, it should be possible to build scalable pattern-storing
systems whose dynamics is governed by quantum laws. Second, we argue with an
example how such Dicke quantum simulators might be used as a solver of "hard"
combinatorial optimization problems.Comment: 5+2 pages, 2 figures. revisited in the exposition and supplementary
added. Comments are welcom
Generalization from correlated sets of patterns in the perceptron
Generalization is a central aspect of learning theory. Here, we propose a
framework that explores an auxiliary task-dependent notion of generalization,
and attempts to quantitatively answer the following question: given two sets of
patterns with a given degree of dissimilarity, how easily will a network be
able to "unify" their interpretation? This is quantified by the volume of the
configurations of synaptic weights that classify the two sets in a similar
manner. To show the applicability of our idea in a concrete setting, we compute
this quantity for the perceptron, a simple binary classifier, using the
classical statistical physics approach in the replica-symmetric ansatz. In this
case, we show how an analytical expression measures the "distance-based
capacity", the maximum load of patterns sustainable by the network, at fixed
dissimilarity between patterns and fixed allowed number of errors. This curve
indicates that generalization is possible at any distance, but with decreasing
capacity. We propose that a distance-based definition of generalization may be
useful in numerical experiments with real-world neural networks, and to explore
computationally sub-dominant sets of synaptic solutions
Devil's staircase phase diagram of the fractional quantum Hall effect in the thin-torus limit
After more than three decades the fractional quantum Hall effect still poses
challenges to contemporary physics. Recent experiments point toward a fractal
scenario for the Hall resistivity as a function of the magnetic field. Here, we
consider the so-called thin-torus limit of the Hamiltonian describing
interacting electrons in a strong magnetic field, restricted to the lowest
Landau level, and we show that it can be mapped onto a one-dimensional lattice
gas with repulsive interactions, with the magnetic field playing the role of a
chemical potential. The statistical mechanics of such models leads to interpret
the sequence of Hall plateaux as a fractal phase diagram, whose landscape shows
a qualitative agreement with experiments.Comment: 5 pages main text, 11 pages supplementary, 2 figure
Current quantization and fractal hierarchy in a driven repulsive lattice gas
Driven lattice gases are widely regarded as the paradigm of collective
phenomena out of equilibrium. While such models are usually studied with
nearest-neighbor interactions, many empirical driven systems are dominated by
slowly decaying interactions such as dipole-dipole and Van der Waals forces.
Motivated by this gap, we study the non-equilibrium stationary state of a
driven lattice gas with slow-decayed repulsive interactions at zero
temperature. By numerical and analytical calculations of the particle current
as a function of the density and of the driving field, we identify (i) an
abrupt breakdown transition between insulating and conducting states, (ii)
current quantization into discrete phases where a finite current flows with
infinite differential resistivity, and (iii) a fractal hierarchy of
excitations, related to the Farey sequences of number theory. We argue that the
origin of these effects is the competition between scales, which also causes
the counterintuitive phenomenon that crystalline states can melt by increasing
the density
Unified Fock space representation of fractional quantum Hall states
Many bosonic (fermionic) fractional quantum Hall states, such as Laughlin,
Moore-Read and Read-Rezayi wavefunctions, belong to a special class of
orthogonal polynomials: the Jack polynomials (times a Vandermonde determinant).
This fundamental observation allows to point out two different recurrence
relations for the coefficients of the permanent (Slater) decomposition of the
bosonic (fermionic) states. Here we provide an explicit Fock space
representation for these wavefunctions by introducing a two-body squeezing
operator which represents them as a Jastrow operator applied to reference
states, which are in general simple periodic one dimensional patterns.
Remarkably, this operator representation is the same for bosons and fermions,
and the different nature of the two recurrence relations is an outcome of
particle statistics.Comment: 10 pages, 3 figure
- …