Deep Drone Racing: From Simulation to Reality with Domain Randomization

Dynamically changing environments, unreliable state estimation, and operation under severe resource constraints are fundamental challenges that limit the deployment of small autonomous drones. We address these challenges in the context of autonomous, vision-based drone racing in dynamic environments. A racing drone must traverse a track with possibly moving gates at high speed. We enable this functionality by combining the performance of a state-of-the-art planning and control system with the perceptual awareness of a convolutional neural network (CNN). The resulting modular system is both platform- and domain-independent: it is trained in simulation and deployed on a physical quadrotor without any fine-tuning. The abundance of simulated data, generated via domain randomization, makes our system robust to changes of illumination and gate appearance. To the best of our knowledge, our approach is the first to demonstrate zero-shot sim-to-real transfer on the task of agile drone flight. We extensively test the precision and robustness of our system, both in simulation and on a physical platform, and show significant improvements over the state of the art.Comment: Accepted as a Regular Paper to the IEEE Transactions on Robotics Journal. arXiv admin note: substantial text overlap with arXiv:1806.0854

SLAM and exploration using differential evolution and fast marching

The exploration and construction of maps in unknown environments is a challenge for robotics. The proposed method is facing this problem by combining effective techniques for planning, SLAM, and a new exploration approach based on the Voronoi Fast Marching method. The final goal of the exploration task is to build a map of the environment that previously the robot did not know. The exploration is not only to determine where the robot should move, but also to plan the movement, and the process of simultaneous localization and mapping. This work proposes the Voronoi Fast Marching method that uses a Fast Marching technique on the Logarithm of the Extended Voronoi Transform of the environment"s image provided by sensors, to determine a motion plan. The Logarithm of the Extended Voronoi Transform imitates the repulsive electric potential from walls and obstacles, and the Fast Marching Method propagates a wave over that potential map. The trajectory is calculated by the gradient method