3,458 research outputs found
kPAM 2.0: Feedback Control for Category-Level Robotic Manipulation
In this paper, we explore generalizable, perception-to-action robotic
manipulation for precise, contact-rich tasks. In particular, we contribute a
framework for closed-loop robotic manipulation that automatically handles a
category of objects, despite potentially unseen object instances and
significant intra-category variations in shape, size and appearance. Previous
approaches typically build a feedback loop on top of a real-time 6-DOF pose
estimator. However, representing an object with a parameterized transformation
from a fixed geometric template does not capture large intra-category shape
variation. Hence we adopt the keypoint-based object representation proposed in
kPAM for category-level pick-and-place, and extend it to closed-loop
manipulation policies with contact-rich tasks. We first augment keypoints with
local orientation information. Using the oriented keypoints, we propose a novel
object-centric action representation in terms of regulating the linear/angular
velocity or force/torque of these oriented keypoints. This formulation is
surprisingly versatile -- we demonstrate that it can accomplish contact-rich
manipulation tasks that require precision and dexterity for a category of
objects with different shapes, sizes and appearances, such as peg-hole
insertion for pegs and holes with significant shape variation and tight
clearance. With the proposed object and action representation, our framework is
also agnostic to the robot grasp pose and initial object configuration, making
it flexible for integration and deployment.Comment: IEEE Robotics and Automation Letter. The video demo is on
https://sites.google.com/view/kpam2
Deep Drone Racing: From Simulation to Reality with Domain Randomization
Dynamically changing environments, unreliable state estimation, and operation
under severe resource constraints are fundamental challenges that limit the
deployment of small autonomous drones. We address these challenges in the
context of autonomous, vision-based drone racing in dynamic environments. A
racing drone must traverse a track with possibly moving gates at high speed. We
enable this functionality by combining the performance of a state-of-the-art
planning and control system with the perceptual awareness of a convolutional
neural network (CNN). The resulting modular system is both platform- and
domain-independent: it is trained in simulation and deployed on a physical
quadrotor without any fine-tuning. The abundance of simulated data, generated
via domain randomization, makes our system robust to changes of illumination
and gate appearance. To the best of our knowledge, our approach is the first to
demonstrate zero-shot sim-to-real transfer on the task of agile drone flight.
We extensively test the precision and robustness of our system, both in
simulation and on a physical platform, and show significant improvements over
the state of the art.Comment: Accepted as a Regular Paper to the IEEE Transactions on Robotics
Journal. arXiv admin note: substantial text overlap with arXiv:1806.0854
- …