44,807 research outputs found
Hi-Val: Iterative Learning of Hierarchical Value Functions for Policy Generation
Task decomposition is effective in manifold applications where the global complexity of a problem makes planning and decision-making too demanding. This is true, for example, in high-dimensional robotics domains, where (1) unpredictabilities and modeling limitations typically prevent the manual specification of robust behaviors, and (2) learning an action policy is challenging due to the curse of dimensionality. In this work, we borrow the concept of Hierarchical Task Networks (HTNs) to decompose the learning procedure, and we exploit Upper Confidence Tree (UCT) search to introduce HOP, a novel iterative algorithm for hierarchical optimistic planning with learned value functions. To obtain better generalization and generate policies, HOP simultaneously learns and uses action values. These are used to formalize constraints within the search space and to reduce the dimensionality of the problem. We evaluate our algorithm both on a fetching task using a simulated 7-DOF KUKA light weight arm and, on a pick and delivery task with a Pioneer robot
Neural principles underlying motor learning and adaptation
Animals, and especially humans, can learn to flexibly adjust their movements to changing environments.
The neural principles underlying this remarkable capability are still not fully understood.
Among the most prominent brain regions controlling movement is primary motor cortex (M1).
Adapted motor behaviour can be related to a change in neural activity within this region. Yet, the
rules guiding this activity change, and thus behavioural adaptation, remain unclear. The overall aim
of this thesis is to investigate the learning process(es) governing the described change in activity in
M1 and, with that, the change in behaviour. Computational modelling is used to study three specific
aspects of learning:
1. What constrains learning to favour some neural activity patterns over others?
2. Can we identify where in a hierarchical pathway learning is happening?
3. How can sensory feedback guide the learning process?
We start by investigating what kind of biological constraints differentially affect learning of new
neural activity that either preserves coactivation patterns between neurons (within-manifold learning),
or requires learning of new coactivation patterns (outside-manifold learning). We propose a new
explanation - the learnability of feedback signals - for why within-manifold activity patterns can be
easier learned than outside-manifold activity patterns. In the second part we develop a hierarchical
model of the motor system to investigate whether we can derive where learning has happened from
only measuring neural activity. Lastly, we investigate how the brain could implement a biologically
plausible learning rule which allows it to correctly assign errors and update recurrent connectivity in
a goal-driven manner.
Overall, our work offers new perspectives on the role of M1 for motor learning and adaptation,
challenges current beliefs, and puts a focus on the role of feedback signals for local plasticity in M1.Open Acces
Hyperbolic Diffusion Embedding and Distance for Hierarchical Representation Learning
Finding meaningful representations and distances of hierarchical data is
important in many fields. This paper presents a new method for hierarchical
data embedding and distance. Our method relies on combining diffusion geometry,
a central approach to manifold learning, and hyperbolic geometry. Specifically,
using diffusion geometry, we build multi-scale densities on the data, aimed to
reveal their hierarchical structure, and then embed them into a product of
hyperbolic spaces. We show theoretically that our embedding and distance
recover the underlying hierarchical structure. In addition, we demonstrate the
efficacy of the proposed method and its advantages compared to existing methods
on graph embedding benchmarks and hierarchical datasets
Hyperbolic Interaction Model For Hierarchical Multi-Label Classification
Different from the traditional classification tasks which assume mutual
exclusion of labels, hierarchical multi-label classification (HMLC) aims to
assign multiple labels to every instance with the labels organized under
hierarchical relations. Besides the labels, since linguistic ontologies are
intrinsic hierarchies, the conceptual relations between words can also form
hierarchical structures. Thus it can be a challenge to learn mappings from word
hierarchies to label hierarchies. We propose to model the word and label
hierarchies by embedding them jointly in the hyperbolic space. The main reason
is that the tree-likeness of the hyperbolic space matches the complexity of
symbolic data with hierarchical structures. A new Hyperbolic Interaction Model
(HyperIM) is designed to learn the label-aware document representations and
make predictions for HMLC. Extensive experiments are conducted on three
benchmark datasets. The results have demonstrated that the new model can
realistically capture the complex data structures and further improve the
performance for HMLC comparing with the state-of-the-art methods. To facilitate
future research, our code is publicly available
Truncated Variational EM for Semi-Supervised Neural Simpletrons
Inference and learning for probabilistic generative networks is often very
challenging and typically prevents scalability to as large networks as used for
deep discriminative approaches. To obtain efficiently trainable, large-scale
and well performing generative networks for semi-supervised learning, we here
combine two recent developments: a neural network reformulation of hierarchical
Poisson mixtures (Neural Simpletrons), and a novel truncated variational EM
approach (TV-EM). TV-EM provides theoretical guarantees for learning in
generative networks, and its application to Neural Simpletrons results in
particularly compact, yet approximately optimal, modifications of learning
equations. If applied to standard benchmarks, we empirically find, that
learning converges in fewer EM iterations, that the complexity per EM iteration
is reduced, and that final likelihood values are higher on average. For the
task of classification on data sets with few labels, learning improvements
result in consistently lower error rates if compared to applications without
truncation. Experiments on the MNIST data set herein allow for comparison to
standard and state-of-the-art models in the semi-supervised setting. Further
experiments on the NIST SD19 data set show the scalability of the approach when
a manifold of additional unlabeled data is available
- …