323 research outputs found
Parameters estimation for spatio-temporal maximum entropy distributions: application to neural spike trains
We propose a numerical method to learn Maximum Entropy (MaxEnt) distributions
with spatio-temporal constraints from experimental spike trains. This is an
extension of two papers [10] and [4] who proposed the estimation of parameters
where only spatial constraints were taken into account. The extension we
propose allows to properly handle memory effects in spike statistics, for large
sized neural networks.Comment: 34 pages, 33 figure
Estimating the Rate-Distortion Function by Wasserstein Gradient Descent
In the theory of lossy compression, the rate-distortion (R-D) function
describes how much a data source can be compressed (in bit-rate) at any given
level of fidelity (distortion). Obtaining for a given data source
establishes the fundamental performance limit for all compression algorithms.
We propose a new method to estimate from the perspective of optimal
transport. Unlike the classic Blahut--Arimoto algorithm which fixes the support
of the reproduction distribution in advance, our Wasserstein gradient descent
algorithm learns the support of the optimal reproduction distribution by moving
particles. We prove its local convergence and analyze the sample complexity of
our R-D estimator based on a connection to entropic optimal transport.
Experimentally, we obtain comparable or tighter bounds than state-of-the-art
neural network methods on low-rate sources while requiring considerably less
tuning and computation effort. We also highlight a connection to
maximum-likelihood deconvolution and introduce a new class of sources that can
be used as test cases with known solutions to the R-D problem.Comment: Accepted as conference paper at NeurIPS 202
Quantum machine learning: a classical perspective
Recently, increased computational power and data availability, as well as
algorithmic advances, have led machine learning techniques to impressive
results in regression, classification, data-generation and reinforcement
learning tasks. Despite these successes, the proximity to the physical limits
of chip fabrication alongside the increasing size of datasets are motivating a
growing number of researchers to explore the possibility of harnessing the
power of quantum computation to speed-up classical machine learning algorithms.
Here we review the literature in quantum machine learning and discuss
perspectives for a mixed readership of classical machine learning and quantum
computation experts. Particular emphasis will be placed on clarifying the
limitations of quantum algorithms, how they compare with their best classical
counterparts and why quantum resources are expected to provide advantages for
learning problems. Learning in the presence of noise and certain
computationally hard problems in machine learning are identified as promising
directions for the field. Practical questions, like how to upload classical
data into quantum form, will also be addressed.Comment: v3 33 pages; typos corrected and references adde
Variations on a theme by Schalkwijk and Kailath
Schalkwijk and Kailath (1966) developed a class of block codes for Gaussian
channels with ideal feedback for which the probability of decoding error
decreases as a second-order exponent in block length for rates below capacity.
This well-known but surprising result is explained and simply derived here in
terms of a result by Elias (1956) concerning the minimum mean-square distortion
achievable in transmitting a single Gaussian random variable over multiple uses
of the same Gaussian channel. A simple modification of the Schalkwijk-Kailath
scheme is then shown to have an error probability that decreases with an
exponential order which is linearly increasing with block length. In the
infinite bandwidth limit, this scheme produces zero error probability using
bounded expected energy at all rates below capacity. A lower bound on error
probability for the finite bandwidth case is then derived in which the error
probability decreases with an exponential order which is linearly increasing in
block length at the same rate as the upper bound.Comment: 18 Pages, 4 figures (added reference
- …