Risk-sensitive Inverse Reinforcement Learning via Semi- and Non-Parametric Methods

The literature on Inverse Reinforcement Learning (IRL) typically assumes that humans take actions in order to minimize the expected value of a cost function, i.e., that humans are risk neutral. Yet, in practice, humans are often far from being risk neutral. To fill this gap, the objective of this paper is to devise a framework for risk-sensitive IRL in order to explicitly account for a human's risk sensitivity. To this end, we propose a flexible class of models based on coherent risk measures, which allow us to capture an entire spectrum of risk preferences from risk-neutral to worst-case. We propose efficient non-parametric algorithms based on linear programming and semi-parametric algorithms based on maximum likelihood for inferring a human's underlying risk measure and cost function for a rich class of static and dynamic decision-making settings. The resulting approach is demonstrated on a simulated driving game with ten human participants. Our method is able to infer and mimic a wide range of qualitatively different driving styles from highly risk-averse to risk-neutral in a data-efficient manner. Moreover, comparisons of the Risk-Sensitive (RS) IRL approach with a risk-neutral model show that the RS-IRL framework more accurately captures observed participant behavior both qualitatively and quantitatively, especially in scenarios where catastrophic outcomes such as collisions can occur.Comment: Submitted to International Journal of Robotics Research; Revision 1: (i) Clarified minor technical points; (ii) Revised proof for Theorem 3 to hold under weaker assumptions; (iii) Added additional figures and expanded discussions to improve readabilit

Optimal Control of Legged-Robots Subject to Friction Cone Constraints

A hierarchical control architecture is presented for energy-efficient control of legged robots subject to variety of linear/nonlinear inequality constraints such as Coulomb friction cones, switching unilateral contacts, actuator saturation limits, and yet minimizing the power losses in the joint actuators. The control formulation can incorporate the nonlinear friction cone constraints into the control without recourse to the common linear approximation of the constraints or introduction of slack variables. A performance metric is introduced that allows trading-off the multiple constraints when otherwise finding an optimal solution is not feasible. Moreover, the projection-based controller does not require the minimal-order dynamics model and hence allows switching contacts that is particularly appealing for legged robots. The fundamental properties of constrained inertia matrix derived are similar to those of general inertia matrix of the system and subsequently these properties are greatly exploited for control design purposes. The problem of task space control with minimum (point-wise) power dissipation subject to all physical constraints is transcribed into a quadratically constrained quadratic programming (QCQP) that can be solved by barrier methods

Analysis and design of Multi-Agent Coverage and Transport algorithms

Els sistemes robòtics multi-agents són sistemes que presenten moltes aplicacions en ciència i enginyeria. En aquest treball estudiarem el control de la cobertura, que es centra en col·locar un grup de sensors per optimitzar la cobertura d’una densitat. Ens centrarem en el cas en què la densitat evoluciona en el temps i estudiarem l’ús de la teoría de perturbacions singulars per resoldre el problema. També considerarem grans eixams de robots, on podem fer servir models continus per analitzar el comportament dels agents. Recentment s'ha proposat models continus que incorporen idees de transport òptim en el problema de transport multi-agent. Presentarem aquests treballs i proveirem algunes modificacions.Los sistemas robóticos multi-agentes son sistemas que presentan muchas aplicaciones en ciencia y ingeniería. En este trabajo vamos a estudiar el control de la cobertura, que se centra en colocar un grupo de sensores para optimizar la cobertura de una densidad. Nos vamos a centrar en el casos en que la densidad evoluciona con el tiempo y estudiaremos el uso de la teoría de perturbaciones singulares para resolver el problema. También consideraremos grandes enjambres de robots, donde podemos utilizar modelos continuos para analizar el comportamiento del enjambre. Recientemente se ha propuesto el uso de modelos continuos que incorporan ideas de transporte òptimo para el problema de transporte multi-agente. Vamos a presentar dichos trabajos y proveeremos algunas modificaciones.Multi-agent robotic systems have shown to be useful and reliable solutions to many problems that arise in science and engineering. In this work we will study Coverage Control, that aims to achieve optimal coverage of a density. We will focus on the case when the density has a time dependence and we will study a Singular Perturbation Theory approach to solve the problem. We will also consider large swarms of agents, where we can develop continuous models to analyze the behaviour of the swarm. Recent work has focused on applying ideas from the theory of Optimal Transport to the Multi-Agent Transport problem. We will review the work and provide some modifications.Outgoin

Batch Policy Learning under Constraints

When learning policies for real-world domains, two important questions arise: (i) how to efficiently use pre-collected off-policy, non-optimal behavior data; and (ii) how to mediate among different competing objectives and constraints. We thus study the problem of batch policy learning under multiple constraints, and offer a systematic solution. We first propose a flexible meta-algorithm that admits any batch reinforcement learning and online learning procedure as subroutines. We then present a specific algorithmic instantiation and provide performance guarantees for the main objective and all constraints. To certify constraint satisfaction, we propose a new and simple method for off-policy policy evaluation (OPE) and derive PAC-style bounds. Our algorithm achieves strong empirical results in different domains, including in a challenging problem of simulated car driving subject to multiple constraints such as lane keeping and smooth driving. We also show experimentally that our OPE method outperforms other popular OPE techniques on a standalone basis, especially in a high-dimensional setting

Proceedings of the 3rd Annual Conference on Aerospace Computational Control, volume 1

Conference topics included definition of tool requirements, advanced multibody component representation descriptions, model reduction, parallel computation, real time simulation, control design and analysis software, user interface issues, testing and verification, and applications to spacecraft, robotics, and aircraft

Proceedings of the NASA Conference on Space Telerobotics, volume 4

Papers presented at the NASA Conference on Space Telerobotics are compiled. The theme of the conference was man-machine collaboration in space. The conference provided a forum for researchers and engineers to exchange ideas on the research and development required for the application of telerobotic technology to the space systems planned for the 1990's and beyond. Volume 4 contains papers related to the following subject areas: manipulator control; telemanipulation; flight experiments (systems and simulators); sensor-based planning; robot kinematics, dynamics, and control; robot task planning and assembly; and research activities at the NASA Langley Research Center

Human-Robot Collaboration in Automotive Assembly

In the past decades, automation in the automobile production line has significantly increased the efficiency and quality of automotive manufacturing. However, in the automotive assembly stage, most tasks are still accomplished manually by human workers because of the complexity and flexibility of the tasks and the high dynamic unconstructed workspace. This dissertation is proposed to improve the level of automation in automotive assembly by human-robot collaboration (HRC). The challenges that eluded the automation in automotive assembly including lack of suitable collaborative robotic systems for the HRC, especially the compact-size high-payload mobile manipulators; teaching and learning frameworks to enable robots to learn the assembly tasks, and how to assist humans to accomplish assembly tasks from human demonstration; task-driving high-level robot motion planning framework to make the trained robot intelligently and adaptively assist human in automotive assembly tasks. The technical research toward this goal has resulted in several peer-reviewed publications. Achievements include: 1) A novel collaborative lift-assist robot for automotive assembly; 2) Approaches of vision-based robot learning of placing tasks from human demonstrations in assembly; 3) Robot learning of assembly tasks and assistance from human demonstrations using Convolutional Neural Network (CNN); 4) Robot learning of assembly tasks and assistance from human demonstrations using Task Constraint-Guided Inverse Reinforcement Learning (TC-IRL); 5) Robot learning of assembly tasks from non-expert demonstrations via Functional Objective-Oriented Network (FOON); 6) Multi-model sampling-based motion planning for trajectory optimization with execution consistency in manufacturing contexts. The research demonstrates the feasibility of a parallel mobile manipulator, which introduces novel conceptions to industrial mobile manipulators for smart manufacturing. By exploring the Robot Learning from Demonstration (RLfD) with both AI-based and model-based approaches, the research also improves robots’ learning capabilities on collaborative assembly tasks for both expert and non-expert users. The research on robot motion planning and control in the dissertation facilitates the safety and human trust in industrial robots in HRC

Robotics 2010

Without a doubt, robotics has made an incredible progress over the last decades. The vision of developing, designing and creating technical systems that help humans to achieve hard and complex tasks, has intelligently led to an incredible variety of solutions. There are barely technical fields that could exhibit more interdisciplinary interconnections like robotics. This fact is generated by highly complex challenges imposed by robotic systems, especially the requirement on intelligent and autonomous operation. This book tries to give an insight into the evolutionary process that takes place in robotics. It provides articles covering a wide range of this exciting area. The progress of technical challenges and concepts may illuminate the relationship between developments that seem to be completely different at first sight. The robotics remains an exciting scientific and engineering field. The community looks optimistically ahead and also looks forward for the future challenges and new development