12,445 research outputs found
Learning modular policies for robotics
A promising idea for scaling robot learning to more complex tasks is to use elemental behaviors as building blocks to compose more complex behavior. Ideally, such building blocks are used in combination with a learning algorithm that is able to learn to select, adapt, sequence and co-activate the building blocks. While there has been a lot of work on approaches that support one of these requirements, no learning algorithm exists that unifies all these properties in one framework. In this paper we present our work on a unified approach for learning such a modular control architecture. We introduce new policy search algorithms that are based on information-theoretic principles and are able to learn to select, adapt and sequence the building blocks. Furthermore, we developed a new representation for the individual building block that supports co-activation and principled ways for adapting the movement. Finally, we summarize our experiments for learning modular control architectures in simulation and with real robots
Towards Task-Prioritized Policy Composition
Combining learned policies in a prioritized, ordered manner is desirable
because it allows for modular design and facilitates data reuse through
knowledge transfer. In control theory, prioritized composition is realized by
null-space control, where low-priority control actions are projected into the
null-space of high-priority control actions. Such a method is currently
unavailable for Reinforcement Learning. We propose a novel, task-prioritized
composition framework for Reinforcement Learning, which involves a novel
concept: The indifferent-space of Reinforcement Learning policies. Our
framework has the potential to facilitate knowledge transfer and modular design
while greatly increasing data efficiency and data reuse for Reinforcement
Learning agents. Further, our approach can ensure high-priority constraint
satisfaction, which makes it promising for learning in safety-critical domains
like robotics. Unlike null-space control, our approach allows learning globally
optimal policies for the compound task by online learning in the
indifference-space of higher-level policies after initial compound policy
construction
Adversarial Discriminative Sim-to-real Transfer of Visuo-motor Policies
Various approaches have been proposed to learn visuo-motor policies for
real-world robotic applications. One solution is first learning in simulation
then transferring to the real world. In the transfer, most existing approaches
need real-world images with labels. However, the labelling process is often
expensive or even impractical in many robotic applications. In this paper, we
propose an adversarial discriminative sim-to-real transfer approach to reduce
the cost of labelling real data. The effectiveness of the approach is
demonstrated with modular networks in a table-top object reaching task where a
7 DoF arm is controlled in velocity mode to reach a blue cuboid in clutter
through visual observations. The adversarial transfer approach reduced the
labelled real data requirement by 50%. Policies can be transferred to real
environments with only 93 labelled and 186 unlabelled real images. The
transferred visuo-motor policies are robust to novel (not seen in training)
objects in clutter and even a moving target, achieving a 97.8% success rate and
1.8 cm control accuracy.Comment: Under review for the International Journal of Robotics Researc
PIC4rl-gym: a ROS2 modular framework for Robots Autonomous Navigation with Deep Reinforcement Learning
Learning agents can optimize standard autonomous navigation improving
flexibility, efficiency, and computational cost of the system by adopting a
wide variety of approaches. This work introduces the \textit{PIC4rl-gym}, a
fundamental modular framework to enhance navigation and learning research by
mixing ROS2 and Gazebo, the standard tools of the robotics community, with Deep
Reinforcement Learning (DRL). The paper describes the whole structure of the
PIC4rl-gym, which fully integrates DRL agent's training and testing in several
indoor and outdoor navigation scenarios and tasks. A modular approach is
adopted to easily customize the simulation by selecting new platforms, sensors,
or models. We demonstrate the potential of our novel gym by benchmarking the
resulting policies, trained for different navigation tasks, with a complete set
of metrics
Inclined Surface Locomotion Strategies for Spherical Tensegrity Robots
This paper presents a new teleoperated spherical tensegrity robot capable of
performing locomotion on steep inclined surfaces. With a novel control scheme
centered around the simultaneous actuation of multiple cables, the robot
demonstrates robust climbing on inclined surfaces in hardware experiments and
speeds significantly faster than previous spherical tensegrity models. This
robot is an improvement over other iterations in the TT-series and the first
tensegrity to achieve reliable locomotion on inclined surfaces of up to
24\degree. We analyze locomotion in simulation and hardware under single and
multi-cable actuation, and introduce two novel multi-cable actuation policies,
suited for steep incline climbing and speed, respectively. We propose
compelling justifications for the increased dynamic ability of the robot and
motivate development of optimization algorithms able to take advantage of the
robot's increased control authority.Comment: 6 pages, 11 figures, IROS 201
- …