97,263 research outputs found
Model Learning for Look-ahead Exploration in Continuous Control
We propose an exploration method that incorporates look-ahead search over
basic learnt skills and their dynamics, and use it for reinforcement learning
(RL) of manipulation policies . Our skills are multi-goal policies learned in
isolation in simpler environments using existing multigoal RL formulations,
analogous to options or macroactions. Coarse skill dynamics, i.e., the state
transition caused by a (complete) skill execution, are learnt and are unrolled
forward during lookahead search. Policy search benefits from temporal
abstraction during exploration, though itself operates over low-level primitive
actions, and thus the resulting policies does not suffer from suboptimality and
inflexibility caused by coarse skill chaining. We show that the proposed
exploration strategy results in effective learning of complex manipulation
policies faster than current state-of-the-art RL methods, and converges to
better policies than methods that use options or parametrized skills as
building blocks of the policy itself, as opposed to guiding exploration. We
show that the proposed exploration strategy results in effective learning of
complex manipulation policies faster than current state-of-the-art RL methods,
and converges to better policies than methods that use options or parameterized
skills as building blocks of the policy itself, as opposed to guiding
exploration.Comment: This is a pre-print of our paper which is accepted in AAAI 201
Hierarchical Visualization of Materials Space with Graph Convolutional Neural Networks
The combination of high throughput computation and machine learning has led
to a new paradigm in materials design by allowing for the direct screening of
vast portions of structural, chemical, and property space. The use of these
powerful techniques leads to the generation of enormous amounts of data, which
in turn calls for new techniques to efficiently explore and visualize the
materials space to help identify underlying patterns. In this work, we develop
a unified framework to hierarchically visualize the compositional and
structural similarities between materials in an arbitrary material space with
representations learned from different layers of graph convolutional neural
networks. We demonstrate the potential for such a visualization approach by
showing that patterns emerge automatically that reflect similarities at
different scales in three representative classes of materials: perovskites,
elemental boron, and general inorganic crystals, covering material spaces of
different compositions, structures, and both. For perovskites, elemental
similarities are learned that reflects multiple aspects of atom properties. For
elemental boron, structural motifs emerge automatically showing characteristic
boron local environments. For inorganic crystals, the similarity and stability
of local coordination environments are shown combining different center and
neighbor atoms. The method could help transition to a data-centered exploration
of materials space in automated materials design.Comment: 22 + 7 pages, 6 + 5 figure
Learning and Transfer of Modulated Locomotor Controllers
We study a novel architecture and training procedure for locomotion tasks. A
high-frequency, low-level "spinal" network with access to proprioceptive
sensors learns sensorimotor primitives by training on simple tasks. This
pre-trained module is fixed and connected to a low-frequency, high-level
"cortical" network, with access to all sensors, which drives behavior by
modulating the inputs to the spinal network. Where a monolithic end-to-end
architecture fails completely, learning with a pre-trained spinal module
succeeds at multiple high-level tasks, and enables the effective exploration
required to learn from sparse rewards. We test our proposed architecture on
three simulated bodies: a 16-dimensional swimming snake, a 20-dimensional
quadruped, and a 54-dimensional humanoid. Our results are illustrated in the
accompanying video at https://youtu.be/sboPYvhpraQComment: Supplemental video available at https://youtu.be/sboPYvhpra
- …