2,354 research outputs found
Risk-sensitive Inverse Reinforcement Learning via Semi- and Non-Parametric Methods
The literature on Inverse Reinforcement Learning (IRL) typically assumes that
humans take actions in order to minimize the expected value of a cost function,
i.e., that humans are risk neutral. Yet, in practice, humans are often far from
being risk neutral. To fill this gap, the objective of this paper is to devise
a framework for risk-sensitive IRL in order to explicitly account for a human's
risk sensitivity. To this end, we propose a flexible class of models based on
coherent risk measures, which allow us to capture an entire spectrum of risk
preferences from risk-neutral to worst-case. We propose efficient
non-parametric algorithms based on linear programming and semi-parametric
algorithms based on maximum likelihood for inferring a human's underlying risk
measure and cost function for a rich class of static and dynamic
decision-making settings. The resulting approach is demonstrated on a simulated
driving game with ten human participants. Our method is able to infer and mimic
a wide range of qualitatively different driving styles from highly risk-averse
to risk-neutral in a data-efficient manner. Moreover, comparisons of the
Risk-Sensitive (RS) IRL approach with a risk-neutral model show that the RS-IRL
framework more accurately captures observed participant behavior both
qualitatively and quantitatively, especially in scenarios where catastrophic
outcomes such as collisions can occur.Comment: Submitted to International Journal of Robotics Research; Revision 1:
(i) Clarified minor technical points; (ii) Revised proof for Theorem 3 to
hold under weaker assumptions; (iii) Added additional figures and expanded
discussions to improve readabilit
Optimal control models of the goal-oriented human locomotion
In recent papers it has been suggested that human locomotion may be modeled
as an inverse optimal control problem. In this paradigm, the trajectories are
assumed to be solutions of an optimal control problem that has to be
determined. We discuss the modeling of both the dynamical system and the cost
to be minimized, and we analyze the corresponding optimal synthesis. The main
results describe the asymptotic behavior of the optimal trajectories as the
target point goes to infinity
ZMP support areas for multi-contact mobility under frictional constraints
We propose a method for checking and enforcing multi-contact stability based
on the Zero-tilting Moment Point (ZMP). The key to our development is the
generalization of ZMP support areas to take into account (a) frictional
constraints and (b) multiple non-coplanar contacts. We introduce and
investigate two kinds of ZMP support areas. First, we characterize and provide
a fast geometric construction for the support area generated by valid contact
forces, with no other constraint on the robot motion. We call this set the full
support area. Next, we consider the control of humanoid robots using the Linear
Pendulum Mode (LPM). We observe that the constraints stemming from the LPM
induce a shrinking of the support area, even for walking on horizontal floors.
We propose an algorithm to compute the new area, which we call pendular support
area. We show that, in the LPM, having the ZMP in the pendular support area is
a necessary and sufficient condition for contact stability. Based on these
developments, we implement a whole-body controller and generate feasible
multi-contact motions where an HRP-4 humanoid locomotes in challenging
multi-contact scenarios.Comment: 14 pages, 10 figure
Motion Planning of Uncertain Ordinary Differential Equation Systems
This work presents a novel motion planning framework, rooted in nonlinear programming theory, that treats uncertain fully and under-actuated dynamical systems described by ordinary differential equations. Uncertainty in multibody dynamical systems comes from various sources, such as: system parameters, initial conditions, sensor and actuator noise, and external forcing. Treatment of uncertainty in design is of paramount practical importance because all real-life systems are affected by it, and poor robustness and suboptimal performance result if it’s not accounted for in a given design. In this work uncertainties are modeled using Generalized Polynomial Chaos and are solved quantitatively using a least-square collocation method. The computational efficiency of this approach enables the inclusion of uncertainty statistics in the nonlinear programming optimization process. As such, the proposed framework allows the user to pose, and answer, new design questions related to uncertain dynamical systems.
Specifically, the new framework is explained in the context of forward, inverse, and hybrid dynamics formulations. The forward dynamics formulation, applicable to both fully and under-actuated systems, prescribes deterministic actuator inputs which yield uncertain state trajectories. The inverse dynamics formulation is the dual to the forward dynamic, and is only applicable to fully-actuated systems; deterministic state trajectories are prescribed and yield uncertain actuator inputs. The inverse dynamics formulation is more computationally efficient as it requires only algebraic evaluations and completely avoids numerical integration. Finally, the hybrid dynamics formulation is applicable to under-actuated systems where it leverages the benefits of inverse dynamics for actuated joints and forward dynamics for unactuated joints; it prescribes actuated state and unactuated input trajectories which yield uncertain unactuated states and actuated inputs.
The benefits of the ability to quantify uncertainty when planning the motion of multibody dynamic systems are illustrated through several case-studies. The resulting designs determine optimal motion plans—subject to deterministic and statistical constraints—for all possible systems within the probability space
- …