87 research outputs found
Value Iteration Networks on Multiple Levels of Abstraction
Learning-based methods are promising to plan robot motion without performing
extensive search, which is needed by many non-learning approaches. Recently,
Value Iteration Networks (VINs) received much interest since---in contrast to
standard CNN-based architectures---they learn goal-directed behaviors which
generalize well to unseen domains. However, VINs are restricted to small and
low-dimensional domains, limiting their applicability to real-world planning
problems.
To address this issue, we propose to extend VINs to representations with
multiple levels of abstraction. While the vicinity of the robot is represented
in sufficient detail, the representation gets spatially coarser with increasing
distance from the robot. The information loss caused by the decreasing
resolution is compensated by increasing the number of features representing a
cell. We show that our approach is capable of solving significantly larger 2D
grid world planning tasks than the original VIN implementation. In contrast to
a multiresolution coarse-to-fine VIN implementation which does not employ
additional descriptive features, our approach is capable of solving challenging
environments, which demonstrates that the proposed method learns to encode
useful information in the additional features. As an application for solving
real-world planning tasks, we successfully employ our method to plan
omnidirectional driving for a search-and-rescue robot in cluttered terrain
Combining Physical Simulators and Object-Based Networks for Control
Physics engines play an important role in robot planning and control;
however, many real-world control problems involve complex contact dynamics that
cannot be characterized analytically. Most physics engines therefore employ .
approximations that lead to a loss in precision. In this paper, we propose a
hybrid dynamics model, simulator-augmented interaction networks (SAIN),
combining a physics engine with an object-based neural network for dynamics
modeling. Compared with existing models that are purely analytical or purely
data-driven, our hybrid model captures the dynamics of interacting objects in a
more accurate and data-efficient manner.Experiments both in simulation and on a
real robot suggest that it also leads to better performance when used in
complex control tasks. Finally, we show that our model generalizes to novel
environments with varying object shapes and materials.Comment: ICRA 2019; Project page: http://sain.csail.mit.ed
Deep Residual Reinforcement Learning
We revisit residual algorithms in both model-free and model-based
reinforcement learning settings. We propose the bidirectional target network
technique to stabilize residual algorithms, yielding a residual version of DDPG
that significantly outperforms vanilla DDPG in the DeepMind Control Suite
benchmark. Moreover, we find the residual algorithm an effective approach to
the distribution mismatch problem in model-based planning. Compared with the
existing TD() method, our residual-based method makes weaker assumptions
about the model and yields a greater performance boost.Comment: AAMAS 202
Propagation Networks for Model-Based Control Under Partial Observation
There has been an increasing interest in learning dynamics simulators for
model-based control. Compared with off-the-shelf physics engines, a learnable
simulator can quickly adapt to unseen objects, scenes, and tasks. However,
existing models like interaction networks only work for fully observable
systems; they also only consider pairwise interactions within a single time
step, both restricting their use in practical systems. We introduce Propagation
Networks (PropNet), a differentiable, learnable dynamics model that handles
partially observable scenarios and enables instantaneous propagation of signals
beyond pairwise interactions. Experiments show that our propagation networks
not only outperform current learnable physics engines in forward simulation,
but also achieve superior performance on various control tasks. Compared with
existing model-free deep reinforcement learning algorithms, model-based control
with propagation networks is more accurate, efficient, and generalizable to
new, partially observable scenes and tasks.Comment: Accepted to ICRA 2019. Project Page: http://propnet.csail.mit.edu
Video: https://youtu.be/ZAxHXegkz4
- …