Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations

Learning agile skills is one of the main challenges in robotics. To this end, reinforcement learning approaches have achieved impressive results. These methods require explicit task information in terms of a reward function or an expert that can be queried in simulation to provide a target control output, which limits their applicability. In this work, we propose a generative adversarial method for inferring reward functions from partial and potentially physically incompatible demonstrations for successful skill acquirement where reference or expert demonstrations are not easily accessible. Moreover, we show that by using a Wasserstein GAN formulation and transitions from demonstrations with rough and partial information as input, we are able to extract policies that are robust and capable of imitating demonstrated behaviors. Finally, the obtained skills such as a backflip are tested on an agile quadruped robot called Solo 8 and present faithful replication of hand-held human demonstrations

Efficient reinforcement learning through variance reduction and trajectory synthesis

Reinforcement learning is a general and unified framework that has been proven promising for many important AI applications, such as robotics, self-driving vehicles. However, current reinforcement learning algorithms suffer from large variance and sampling inefficiency, which leads to slow convergent rate as well as unstable performance. In this thesis, we manage to alleviate these two relevant problems. For enormous variance, we combine variance reduced optimization with deep Q-learning. For inefficient sampling, we propose novel framework that integrates self-imitation learning and artificial synthesis procedure. Our approaches, which are flexible and could be extended to many tasks, prove their effectiveness through experiments on Atari and MuJoCo environment

Accelerated Risk Assessment And Domain Adaptation For Autonomous Vehicles

Autonomous vehicles (AVs) are already driving on public roads around the US; however, their rate of deployment far outpaces quality assurance and regulatory efforts. Consequently, even the most elementary tasks, such as automated lane keeping, have not been certified for safety, and operations are constrained to narrow domains. First, due to the limitations of worst-case analysis techniques, we hypothesize that new methods must be developed to quantify and bound the risk of AVs. Counterintuitively, the better the performance of the AV under consideration, the harder it is to accurately estimate its risk as failures become rare and difficult to sample. This thesis presents a new estimation procedure and framework that can efficiently evaluate and AV\u27s risk even in the rare event regime. We demonstrate the approach\u27s performance on a variety of AV software stacks. Second, given a framework for AV evaluation, we turn to a related question: how can AV software be efficiently adapted for new or expanded operating conditions? We hypothesize that stochastic search techniques can improve the naive trial-and-error approach commonly used today. One of the most challenging aspects of this task is that proficient driving requires making tradeoffs between performance and safety. Moreover, for novel scenarios or operational domains there may be little data that can be used to understand the behavior of other drivers. To study these challenges we create a low-cost scale platform, simulator, benchmarks, and baseline solutions. Using this testbed, we develop a new population-based self-play method for creating dynamic actors and detail both offline and online procedures for adapting AV components to these conditions. Taken as a whole, this work represents a rigorous approach to the evaluation and improvement of AV software

Video Understanding: A Predictive Analytics Perspective

This dissertation includes a detailed study of video predictive understanding, an emerging perspective on video-based computer vision research. This direction explores machine vision techniques to fill in missing spatiotemporal information in videos (e.g., predict the future), which is of great importance for understanding real world dynamics and benefits many applications. We investigate this direction with depth and breadth. Four emerging areas are considered and improved by our efforts: early action recognition, future activity prediction, trajectory prediction and procedure planning. For each, our research presents innovative solutions based on machine learning techniques (deep learning in particular) and meanwhile pays special attention to their interpretability, multi-modality and efficiency, which we consider as critical for next-generation Artificial Intelligence (AI). Finally, we conclude this dissertation by discussing current shortcomings as well as future directions

Video Understanding: A Predictive Analytics Perspective

This dissertation includes a detailed study of video predictive understanding, an emerging perspective on video-based computer vision research. This direction explores machine vision techniques to fill in missing spatiotemporal information in videos (e.g., predict the future), which is of great importance for understanding real world dynamics and benefits many applications. We investigate this direction with depth and breadth. Four emerging areas are considered and improved by our efforts: early action recognition, future activity prediction, trajectory prediction and procedure planning. For each, our research presents innovative solutions based on machine learning techniques (deep learning in particular) and meanwhile pays special attention to their interpretability, multi-modality and efficiency, which we consider as critical for next-generation Artificial Intelligence (AI). Finally, we conclude this dissertation by discussing current shortcomings as well as future directions