Effective Target Aware Visual Navigation for UAVs

In this paper we propose an effective vision-based navigation method that allows a multirotor vehicle to simultaneously reach a desired goal pose in the environment while constantly facing a target object or landmark. Standard techniques such as Position-Based Visual Servoing (PBVS) and Image-Based Visual Servoing (IBVS) in some cases (e.g., while the multirotor is performing fast maneuvers) do not allow to constantly maintain the line of sight with a target of interest. Instead, we compute the optimal trajectory by solving a non-linear optimization problem that minimizes the target re-projection error while meeting the UAV's dynamic constraints. The desired trajectory is then tracked by means of a real-time Non-linear Model Predictive Controller (NMPC): this implicitly allows the multirotor to satisfy both the required constraints. We successfully evaluate the proposed approach in many real and simulated experiments, making an exhaustive comparison with a standard approach.Comment: Conference paper at "European Conference on Mobile Robotics" (ECMR) 201

Towards Autonomous Aviation Operations: What Can We Learn from Other Areas of Automation?

Rapid advances in automation has disrupted and transformed several industries in the past 25 years. Automation has evolved from regulation and control of simple systems like controlling the temperature in a room to the autonomous control of complex systems involving network of systems. The reason for automation varies from industry to industry depending on the complexity and benefits resulting from increased levels of automation. Automation may be needed to either reduce costs or deal with hazardous environment or make real-time decisions without the availability of humans. Space autonomy, Internet, robotic vehicles, intelligent systems, wireless networks and power systems provide successful examples of various levels of automation. NASA is conducting research in autonomy and developing plans to increase the levels of automation in aviation operations. This paper provides a brief review of levels of automation, previous efforts to increase levels of automation in aviation operations and current level of automation in the various tasks involved in aviation operations. It develops a methodology to assess the research and development in modeling, sensing and actuation needed to advance the level of automation and the benefits associated with higher levels of automation. Section II describes provides an overview of automation and previous attempts at automation in aviation. Section III provides the role of automation and lessons learned in Space Autonomy. Section IV describes the success of automation in Intelligent Transportation Systems. Section V provides a comparison between the development of automation in other areas and the needs of aviation. Section VI provides an approach to achieve increased automation in aviation operations based on the progress in other areas. The final paper will provide a detailed analysis of the benefits of increased automation for the Traffic Flow Management (TFM) function in aviation operations

Search-Based Task and Motion Planning for Hybrid Systems: Agile Autonomous Vehicles

To achieve optimal robot behavior in dynamic scenarios we need to consider complex dynamics in a predictive manner. In the vehicle dynamics community, it is well know that to achieve time-optimal driving on low surface, the vehicle should utilize drifting. Hence many authors have devised rules to split circuits and employ drifting on some segments. These rules are suboptimal and do not generalize to arbitrary circuit shapes (e.g., S-like curves). So, the question "When to go into which mode and how to drive in it?" remains unanswered. To choose the suitable mode (discrete decision), the algorithm needs information about the feasibility of the continuous motion in that mode. This makes it a class of Task and Motion Planning (TAMP) problems, which are known to be hard to solve optimally in real-time. In the AI planning community, search methods are commonly used. However, they cannot be directly applied to TAMP problems due to the continuous component. Here, we present a search-based method that effectively solves this problem and efficiently searches in a highly dimensional state space with nonlinear and unstable dynamics. The space of the possible trajectories is explored by sampling different combinations of motion primitives guided by the search. Our approach allows to use multiple locally approximated models to generate motion primitives (e.g., learned models of drifting) and effectively simplify the problem without losing accuracy. The algorithm performance is evaluated in simulated driving on a mixed-track with segments of different curvatures (right and left). Our code is available at https://git.io/JenvBComment: Accepted to the journal Engineering Applications of Artificial Intelligence; 19 pages, 18 figures, code: https://git.io/JenvB. arXiv admin note: text overlap with arXiv:1907.0782

Deep Drone Racing: From Simulation to Reality with Domain Randomization

Dynamically changing environments, unreliable state estimation, and operation under severe resource constraints are fundamental challenges that limit the deployment of small autonomous drones. We address these challenges in the context of autonomous, vision-based drone racing in dynamic environments. A racing drone must traverse a track with possibly moving gates at high speed. We enable this functionality by combining the performance of a state-of-the-art planning and control system with the perceptual awareness of a convolutional neural network (CNN). The resulting modular system is both platform- and domain-independent: it is trained in simulation and deployed on a physical quadrotor without any fine-tuning. The abundance of simulated data, generated via domain randomization, makes our system robust to changes of illumination and gate appearance. To the best of our knowledge, our approach is the first to demonstrate zero-shot sim-to-real transfer on the task of agile drone flight. We extensively test the precision and robustness of our system, both in simulation and on a physical platform, and show significant improvements over the state of the art.Comment: Accepted as a Regular Paper to the IEEE Transactions on Robotics Journal. arXiv admin note: substantial text overlap with arXiv:1806.0854

High fidelity progressive reinforcement learning for agile maneuvering UAVs

In this work, we present a high fidelity model based progressive reinforcement learning method for control system design for an agile maneuvering UAV. Our work relies on a simulation-based training and testing environment for doing software-in-the-loop (SIL), hardware-in-the-loop (HIL) and integrated flight testing within photo-realistic virtual reality (VR) environment. Through progressive learning with the high fidelity agent and environment models, the guidance and control policies build agile maneuvering based on fundamental control laws. First, we provide insight on development of high fidelity mathematical models using frequency domain system identification. These models are later used to design reinforcement learning based adaptive flight control laws allowing the vehicle to be controlled over a wide range of operating conditions covering model changes on operating conditions such as payload, voltage and damage to actuators and electronic speed controllers (ESCs). We later design outer flight guidance and control laws. Our current work and progress is summarized in this work

Fast, Autonomous Flight in GPS-Denied and Cluttered Environments

One of the most challenging tasks for a flying robot is to autonomously navigate between target locations quickly and reliably while avoiding obstacles in its path, and with little to no a-priori knowledge of the operating environment. This challenge is addressed in the present paper. We describe the system design and software architecture of our proposed solution, and showcase how all the distinct components can be integrated to enable smooth robot operation. We provide critical insight on hardware and software component selection and development, and present results from extensive experimental testing in real-world warehouse environments. Experimental testing reveals that our proposed solution can deliver fast and robust aerial robot autonomous navigation in cluttered, GPS-denied environments.Comment: Pre-peer reviewed version of the article accepted in Journal of Field Robotic