Deep Drone Racing: From Simulation to Reality with Domain Randomization

Dynamically changing environments, unreliable state estimation, and operation under severe resource constraints are fundamental challenges that limit the deployment of small autonomous drones. We address these challenges in the context of autonomous, vision-based drone racing in dynamic environments. A racing drone must traverse a track with possibly moving gates at high speed. We enable this functionality by combining the performance of a state-of-the-art planning and control system with the perceptual awareness of a convolutional neural network (CNN). The resulting modular system is both platform- and domain-independent: it is trained in simulation and deployed on a physical quadrotor without any fine-tuning. The abundance of simulated data, generated via domain randomization, makes our system robust to changes of illumination and gate appearance. To the best of our knowledge, our approach is the first to demonstrate zero-shot sim-to-real transfer on the task of agile drone flight. We extensively test the precision and robustness of our system, both in simulation and on a physical platform, and show significant improvements over the state of the art.Comment: Accepted as a Regular Paper to the IEEE Transactions on Robotics Journal. arXiv admin note: substantial text overlap with arXiv:1806.0854

Learning and planning in videogames via task decomposition

Artificial intelligence (AI) methods have come a long way in tabletop games, with computer programs having now surpassed human experts in the challenging games of chess, Go and heads-up no-limit Texas hold'em. However, a significant simplifying factor in these games is that individual decisions have a relatively large impact on the state of the game. The real world, however, is granular. Human beings are continually presented with new information and are faced with making a multitude of tiny decisions every second. Viewed in these terms, feedback is often sparse, meaning that it only arrives after one has made a great number of decisions. Moreover, in many real-world problems there is a continuous range of actions to choose from, and attaining meaningful feedback from the environment often requires a strong degree of action coordination. Videogames, in which players must likewise contend with granular time scales and continuous action spaces, are in this sense a better proxy for real-world problems, and have thus become regarded by many as the new frontier in games AI. Seemingly, the way in which human players approach granular decision-making in videogames is by decomposing complex tasks into high-level subproblems, thereby allowing them to focus on the "big picture". For example, in Super Mario World, human players seem to look ahead in extended steps, such as climbing a vine or jumping over a pit, rather than planning one frame at a time. Currently though, this type of reasoning does not come easily to machines, leaving many open research problems related to task decomposition. This thesis focuses on three such problems in particular: (1) The challenge of learning subgoals autonomously, so as to lessen the issue of sparse feedback. (2) The challenge of combining discrete planning techniques with extended actions whose durations and effects on the environment are uncertain. (3) The questions of when and why it is beneficial to reason over high-level continuous control variables, such as the velocity of a player-controlled ship, rather than over the most low-level actions available. We address these problems via new algorithms and novel experimental design, demonstrating empirically that our algorithms are more efficient than strong baselines that do not leverage task decomposition, and yielding insight into the types of environment where task decomposition is likely to be beneficial

Behavior based autonomous mobile Robot for industrial logistics

The design of robot behaviors to meet the requirements of the new industrial era - Industry 4.0 - has grown significantly in recent years. Especially the demand for flexible and adaptable systems has increased exponentially since intelligent robots started to be integrated into assembly lines and replace human activities. Tools such as Finite State Machines have proven to be an understandable and quick way to solve high-level problems in robotics; however, unmanageable when complexity rises. They become confusing and unreadable, making their modification and mainte- nance a problem. New tools, such as Behavior Trees, have emerged, creating modular, flexible, and adaptable systems without sacrificing readability with the increased com- plexity. The proposed architecture follows a hierarchical layered approach taking advantage of Behavior Trees, developing modular robot skills and system interfaces to create an autonomous behavior-based system. The software was implemented and tested in an Autonomous Mobile Robot capable of navigating complex environments and executing basic tasks. The results showed real advantages in using the layer-based approach, particularly giving the system modularity and increased flexibility capable of being easily improved and used in other systems. It was also concluded that Behavior Trees are an adequate tool for reactive systems in highly dynamic environments.Nos últimos anos, tem-se verificado um crescimentos na modelação de comportamen- tos robóticos com o objetivo de satisfazer necessidades dos novos paradigmas da indústria. Em particular, na indústria 4.0, com a integração de robôs nas linhas de produção e a subs- tituição dos humanos em diversas atividades, tem-se verificado um aumento na exigência de sistemas mais adaptáveis e flexíveis. Ferramentas tais como as máquinas de estado provaram ser percetíveis e de fácil uti- lização na resolução de problemas na área da robótica. No entanto, com o aumento da complexidade, tornam-se problemáticas pela sua desorganização e ilegibilidade. Por con- seguinte, emergiram novas estruturas, tais como as árvores de comportamento, capazes de tornar os sistemas mais modulares e flexíveis. A arquitetura por hierarquisação de camadas proposta, tira partido das vantagens das árvores de comportamento, com o desenvolvimento de comportamentos e interfaces de modo a criar um sistema reativo e autónomo. O software foi implementado e testado num robô móvel autónomo, capaz de navegar em ambientes complexos e de executar tarefas basicas. Os resultados mostraram vantagens na utilização da arquitetura proposta, em parti- cular, trazendo modularidade e flexibilidade ao sistema robótico, permitindo uma futura melhoria de cada um dos módulos, tal como, a sua utilização noutros sistemas