175,614 research outputs found
Low-Dimensional State and Action Representation Learning with MDP Homomorphism Metrics
Deep Reinforcement Learning has shown its ability in solving complicated problems directly from high-dimensional observations. However, in end-to-end settings, Reinforcement Learning algorithms are not sample-efficient and requires long training times and quantities of data. In this work, we proposed a framework for sample-efficient Reinforcement Learning that take advantage of state and action representations to transform a high-dimensional problem into a low-dimensional one. Moreover, we seek to find the optimal policy mapping latent states to latent actions. Because now the policy is learned on abstract representations, we enforce, using auxiliary loss functions, the lifting of such policy to the original problem domain. Results show that the novel framework can efficiently learn low-dimensional and interpretable state and action representations and the optimal latent policy
Error Metrics for Learning Reliable Manifolds from Streaming Data
Spectral dimensionality reduction is frequently used to identify
low-dimensional structure in high-dimensional data. However, learning
manifolds, especially from the streaming data, is computationally and memory
expensive. In this paper, we argue that a stable manifold can be learned using
only a fraction of the stream, and the remaining stream can be mapped to the
manifold in a significantly less costly manner. Identifying the transition
point at which the manifold is stable is the key step. We present error metrics
that allow us to identify the transition point for a given stream by
quantitatively assessing the quality of a manifold learned using Isomap. We
further propose an efficient mapping algorithm, called S-Isomap, that can be
used to map new samples onto the stable manifold. We describe experiments on a
variety of data sets that show that the proposed approach is computationally
efficient without sacrificing accuracy
Coordinated Local Metric Learning
International audienceMahalanobis metric learning amounts to learning a linear data projection, after which the L2 metric is used to compute distances. To allow more flexible metrics, not restricted to linear projections, local metric learning techniques have been developed. Most of these methods partition the data space using clustering, and for each cluster a separate metric is learned. Using local metrics, however, it is not clear how to measure distances between data points assigned to different clusters. In this paper we propose to embed the local metrics in a global low-dimensional representation, in which the L2 metric can be used. With each cluster we associate a linear mapping that projects the data to the global representation. This global representation directly allows computing distances between points regardless to which local cluster they belong. Moreover, it also enables data visualization in a single view, and the use of L2 based efficient retrieval methods. Experiments on the Labeled Faces in the Wild dataset show that our approach improves over previous global and local metric learning approaches
A Kernel Perspective on Behavioural Metrics for Markov Decision Processes
Behavioural metrics have been shown to be an effective mechanism for
constructing representations in reinforcement learning. We present a novel
perspective on behavioural metrics for Markov decision processes via the use of
positive definite kernels. We leverage this new perspective to define a new
metric that is provably equivalent to the recently introduced MICo distance
(Castro et al., 2021). The kernel perspective further enables us to provide new
theoretical results, which has so far eluded prior work. These include bounding
value function differences by means of our metric, and the demonstration that
our metric can be provably embedded into a finite-dimensional Euclidean space
with low distortion error. These are two crucial properties when using
behavioural metrics for reinforcement learning representations. We complement
our theory with strong empirical results that demonstrate the effectiveness of
these methods in practice.Comment: Published in TML
Mosquito Detection with Neural Networks: The Buzz of Deep Learning
Many real-world time-series analysis problems are characterised by scarce
data. Solutions typically rely on hand-crafted features extracted from the time
or frequency domain allied with classification or regression engines which
condition on this (often low-dimensional) feature vector. The huge advances
enjoyed by many application domains in recent years have been fuelled by the
use of deep learning architectures trained on large data sets. This paper
presents an application of deep learning for acoustic event detection in a
challenging, data-scarce, real-world problem. Our candidate challenge is to
accurately detect the presence of a mosquito from its acoustic signature. We
develop convolutional neural networks (CNNs) operating on wavelet
transformations of audio recordings. Furthermore, we interrogate the network's
predictive power by visualising statistics of network-excitatory samples. These
visualisations offer a deep insight into the relative informativeness of
components in the detection problem. We include comparisons with conventional
classifiers, conditioned on both hand-tuned and generic features, to stress the
strength of automatic deep feature learning. Detection is achieved with
performance metrics significantly surpassing those of existing algorithmic
methods, as well as marginally exceeding those attained by individual human
experts.Comment: For data and software related to this paper, see
http://humbug.ac.uk/kiskin2017/. Submitted as a conference paper to ECML 201
- …