Improving Grasp Skills Using Schema Structured Learning

Abstract In the control-based approach to robotics, complex behavior is created by sequencing and combining control primitives. While it is desirable for the robot to autonomously learn the correct control sequence, searching through the large number of potential solutions can be time consuming. This paper constrains this search to variations of a generalized solution encoded in a framework known as an action schema. A new algorithm, SCHEMA STRUCTURED LEARNING, is proposed that repeatedly executes variations of the generalized solution in search of instantiations that satisfy action schema objectives. This approach is tested in a grasping task where Dexter, the UMass humanoid robot, learns which reaching and grasping controllers maximize the probability of grasp success

Inverse-Dynamics MPC via Nullspace Resolution

Optimal control (OC) using inverse dynamics provides numerical benefits such as coarse optimization, cheaper computation of derivatives, and a high convergence rate. However, in order to take advantage of these benefits in model predictive control (MPC) for legged robots, it is crucial to handle its large number of equality constraints efficiently. To accomplish this, we first (i) propose a novel approach to handle equality constraints based on nullspace parametrization. Our approach balances optimality, and both dynamics and equality-constraint feasibility appropriately, which increases the basin of attraction to good local minima. To do so, we then (ii) adapt our feasibility-driven search by incorporating a merit function. Furthermore, we introduce (iii) a condensed formulation of the inverse dynamics that considers arbitrary actuator models. We also develop (iv) a novel MPC based on inverse dynamics within a perception locomotion framework. Finally, we present (v) a theoretical comparison of optimal control with the forward and inverse dynamics, and evaluate both numerically. Our approach enables the first application of inverse-dynamics MPC on hardware, resulting in state-of-the-art dynamic climbing on the ANYmal robot. We benchmark it over a wide range of robotics problems and generate agile and complex maneuvers. We show the computational reduction of our nullspace resolution and condensed formulation (up to 47.3%). We provide evidence of the benefits of our approach by solving coarse optimization problems with a high convergence rate (up to 10 Hz of discretization). Our algorithm is publicly available inside CROCODDYL.Comment: 17 pages, 14 figures, under-revie

Motion primitive based random planning for loco-manipulation tasks

Several advanced control laws are available for complex robotic systems such as humanoid robots and mobile manipulators. Controls are usually developed for locomotion or for manipulation purposes. Resulting motions are usually executed sequentially and the potentiality of the robotic platform is not fully exploited. In this work we consider the problem of loco-manipulation planning for a robot with given parametrized control laws known as primitives. Such primitives, may have not been designed to be executed simultaneously and by composing them instability may easily arise. With the proposed approach, primitives combination that guarantee stability of the system are obtained resulting in complex whole-body behavior. A formal definition of motion primitives is provided and a random sampling approach on a manifold with limited dimension is investigated. Probabilistic completeness and asymptotic optimality are also proved. The proposed approach is tested both on a mobile manipulator and on the humanoid robot Walk-Man, performing loco-manipulation tasks

Tactile Gloves for Autonomous Grasping With the NASA/DARPA Robonaut

Tactile data from rugged gloves are providing the foundation for developing autonomous grasping skills for the NASA/DARPA Robonaut, a dexterous humanoid robot. These custom gloves compliment the human like dexterity available in the Robonaut hands. Multiple versions of the gloves are discussed, showing a progression in using advanced materials and construction techniques to enhance sensitivity and overall sensor coverage. The force data provided by the gloves can be used to improve dexterous, tool and power grasping primitives. Experiments with the latest gloves focus on the use of tools, specifically a power drill used to approximate an astronaut's torque tool

Biologically-Inspired 3D Grasp Synthesis Based on Visual Exploration

Object grasping is a typical human ability which is widely studied from both a biological and an engineering point of view. This paper presents an approach to grasp synthesis inspired by the human neurophysiology of actionoriented vision. Our grasp synthesis method is built upon an architecture which, taking into account the differences between robotic and biological systems, proposes an adaptation of brain models to the peculiarities of robotic setups. The architecture modularity allows for scalability and integration of complex robotic tasks. The grasp synthesis is designed as integrated with the extraction of a 3D object description, so that the object visual analysis is actively driven by the needs of the grasp synthesis: visual reconstruction is performed incrementally and selectively on the regions of the object that are considered more interesting for graspin

Control of Cooperating Mobile Manipulators

We describe a framework and control algorithms for coordinating multiple mobile robots with manipulators focusing on tasks that require grasping, manipulation and transporting large and possibly flexible objects without special purpose fixtures. Because each robot has an independent controller and is autonomous, the coordination and synergy are realized through sensing and communication. The robots can cooperatively transport objects and march in a tightly controlled formation, while also having the capability to navigate autonomously. We describe the key aspects of the overall hierarchy and the basic algorithms, with specific applications to our experimental testbed consisting of three robots. We describe results from many experiments that demonstrate the ability of the system to carry flexible boards and large boxes as well as the system’s robustness to alignment and odometry errors

UAV perception for safe flight under physical interaction

The control of autonomous flying vehicles with navigation purposes is a challenging task. Complexity arises mainly due to the non-linearity and uncertainty inherently present in the flight mechanics and aircraft-air interactions. Recently, interest has grown for equipping unmanned vehicles with the capacity to interact with their environment, other vehicles or humans. This will enable interesting applications such as autonomous load carrying, aerial refueling or parcel delivering. Having measured the interaction wrenches ease the control problem which can be configured to reject disturbances or to take profit of them to fulfill mission objectives. This thesis will contribute to this area by providing perception solutions which use limited and low cost sensors that enable state and disturbance estimation for possible, but not restricted to, interaction scenarios. This thesis contain three parts. The first part, introduces basic concepts related to the navigation state, aircraft dynamics, and sensor models. In addition, the platform under study is presented and mathematical models associated to it are calibrated. The second part is devoted to the observability analysis and the design of state observers. Linear and non-linear observability analysis techniques are used to unveil that the state of quadrotors equipped with GPS, magnetometers an IMU sensors cannot be uniquely identified in some specific flight configurations. Results of this section are relevant because the conflicting flight configurations contain hover, a flight maneuverer central in many unmanned aerial missions of VtoL vehicles. For many possible singular configurations, insightful descriptions and interpretations of the solution space known as indistinguishable region is provided. Findings are verified in simulation scenarios where it can be seen how a filter fails to recover the true state of an aircraft when imposing the hover flight condition. We discuss then the design of Extended Kalman Filters for state estimation that considers the available sensors. Issues that are typically not reported in the literature, such as when to update or propagate in the estimator algorithm or which coordinate frame should be used to represent each state variable are discussed. This leads to the formulation of four potentially equivalent but different discrete event-based filters for which precise algorithmic expressions are given. We compare the results of the four filters in simulation under known favorable conditions for observability. In order to diminish the effect of flying in the conflicting observability configurations, we provide an alternative filter based on the Schmidt Kalman Filter (SKF). The proposed filter shares the structure of the EKF, behaves better in the instants that the EKF fails and provides similar results in the remaining conditions. The last part of the thesis deals with the estimation of external disturbances. Disturbance estimation results are based on the derivation of a linear model for the aircraft dynamics which then extended with a high order disturbance model to enable the estimation of fast varying disturbances. Two external disturbance estimators from the literature are reviewed and adapted to the new model. Also, two Kalman observers that exploit the linearity of the derived model are presented. A simulation comparison is provided demonstrating that the KF disturbance estimators outperform the other. In addition, this part presents a design methodology of generic quadratic bounded observers for linear systems with ellipsoidal bounded uncertainty. The derived observers maximize a user tunable compromise between the estimation convergence speed and the final volume containing the estimation error. An observer for disturbances acting on a flying platform is derived considering the high order disturbance model above mentioned. Finally, an analysis of the estimation performance with respect to the design parameters is presented.Esta tesis, contribuye en este área formulando soluciones de percepción que permiten la estimación del estado y perturbaciones externas en condiciones normales de vuelos así como casos de interacción para UAVs equipados con sensores limitados y de bajo coste. La tesis se estructura en tres partes. La primera de ellas introduce los conceptos básicos relacionados con el estado de navegación, la dinámica de la aeronave y modelos de sensores. Además, se presenta la plataforma de estudio así como los modelos matemáticos asociados a ella y su calibración. La segunda parte está destinada al análisis de observabilidad y el diseño de observadores de estado. Los resultados de esta sección son importantes porque dentro de las condiciones de vuelo conflictivas se encuentra el vuelo a punto fijo, una maniobra de vuelo central en muchas misiones de vehículos VToL. Se analizan estas condiciones críticas de vuelo y para ellas se deriva y describe el espacio de soluciones posible conocido como región indistinguible. Los resultados son verificados en simulación dónde se puede apreciar como un estimador de estado falla al intentar realizar su tarea cuando la aeronave está en vuelo a punto fijo. Seguidamente se presenta el diseño de filtros extendidos de Kalman (EKF) que proveen estimaciones del estado con la información limitada de los sensores disponibles. Se discuten conceptos que habitualmente no se presentan en la literatura como cuando actualizar o propagar en el algoritmo de estimación o que sistema de referencia se debe utilizar para representar adecuadamente las variables de estado. Esto lleva a la formulación algorítmica de cuatro filtros discretos basados en eventos, diferentes, pero en esencia equivalentes. Se derivan rutinas de inicialización para los filtros y se comparan los resultados en simulación bajo condiciones favorables de estimación. Con la idea de disminuir el efecto de volar en configuraciones de observabilidad conflictivas, se deriva un filtro alternativo basado en el filtro de Schmidt Kalman (SKF). El filtro propuesto comparte estructura con el EKF, tiene un mejor comportamiento allí dónde le EKF falla y una respuesta similar en el resto de condiciones de vuelo. La última parte de la tesis trata con la estimación de perturbaciones externas. Para ello se deriva un modelo lineal que relaciona fuerzas y momentos con velocidades junto a un modelo de alto orden para las perturbaciones. Se estudia su aplicación a dos modelos para la estimación de perturbaciones ya presentes en la literatura. Además, se proponen dos nuevos filtros de Kalman que se aprovechan de la linealidad del modelo. Se presenta una comparativa basada en la simulación de escenarios ideales así como realistas que demuestra que los filtros KF superan al resto. Esta misma parte de la tesis presenta el diseño genérico de estimadores "quadratic bounded" para sistemas dinámicos lineales cuya incertidumbre se encuentra acotada dentro de elipsoides. Estos estimadores maximizan un compromiso, ajustable por el usuario que contempla la velocidad de convergencia así como el volumen de la solución final que contiene el error de estimación. Se deriva un observador de perturbaciones para plataformas aéreas basado en el modelo de alto orden arriba mencionado. Finalmente, se presenta un análisis del desempeño de estimación en función de los parámetros de diseño del filtro

UAV perception for safe flight under physical interaction

Aplicat embargament des de la data de defensa fins al maig 2020The control of autonomous flying vehicles with navigation purposes is a challenging task. Complexity arises mainly due to the non-linearity and uncertainty inherently present in the flight mechanics and aircraft-air interactions. Recently, interest has grown for equipping unmanned vehicles with the capacity to interact with their environment, other vehicles or humans. This will enable interesting applications such as autonomous load carrying, aerial refueling or parcel delivering. Having measured the interaction wrenches ease the control problem which can be configured to reject disturbances or to take profit of them to fulfill mission objectives. This thesis will contribute to this area by providing perception solutions which use limited and low cost sensors that enable state and disturbance estimation for possible, but not restricted to, interaction scenarios. This thesis contain three parts. The first part, introduces basic concepts related to the navigation state, aircraft dynamics, and sensor models. In addition, the platform under study is presented and mathematical models associated to it are calibrated. The second part is devoted to the observability analysis and the design of state observers. Linear and non-linear observability analysis techniques are used to unveil that the state of quadrotors equipped with GPS, magnetometers an IMU sensors cannot be uniquely identified in some specific flight configurations. Results of this section are relevant because the conflicting flight configurations contain hover, a flight maneuverer central in many unmanned aerial missions of VtoL vehicles. For many possible singular configurations, insightful descriptions and interpretations of the solution space known as indistinguishable region is provided. Findings are verified in simulation scenarios where it can be seen how a filter fails to recover the true state of an aircraft when imposing the hover flight condition. We discuss then the design of Extended Kalman Filters for state estimation that considers the available sensors. Issues that are typically not reported in the literature, such as when to update or propagate in the estimator algorithm or which coordinate frame should be used to represent each state variable are discussed. This leads to the formulation of four potentially equivalent but different discrete event-based filters for which precise algorithmic expressions are given. We compare the results of the four filters in simulation under known favorable conditions for observability. In order to diminish the effect of flying in the conflicting observability configurations, we provide an alternative filter based on the Schmidt Kalman Filter (SKF). The proposed filter shares the structure of the EKF, behaves better in the instants that the EKF fails and provides similar results in the remaining conditions. The last part of the thesis deals with the estimation of external disturbances. Disturbance estimation results are based on the derivation of a linear model for the aircraft dynamics which then extended with a high order disturbance model to enable the estimation of fast varying disturbances. Two external disturbance estimators from the literature are reviewed and adapted to the new model. Also, two Kalman observers that exploit the linearity of the derived model are presented. A simulation comparison is provided demonstrating that the KF disturbance estimators outperform the other. In addition, this part presents a design methodology of generic quadratic bounded observers for linear systems with ellipsoidal bounded uncertainty. The derived observers maximize a user tunable compromise between the estimation convergence speed and the final volume containing the estimation error. An observer for disturbances acting on a flying platform is derived considering the high order disturbance model above mentioned. Finally, an analysis of the estimation performance with respect to the design parameters is presented.Esta tesis, contribuye en este área formulando soluciones de percepción que permiten la estimación del estado y perturbaciones externas en condiciones normales de vuelos así como casos de interacción para UAVs equipados con sensores limitados y de bajo coste. La tesis se estructura en tres partes. La primera de ellas introduce los conceptos básicos relacionados con el estado de navegación, la dinámica de la aeronave y modelos de sensores. Además, se presenta la plataforma de estudio así como los modelos matemáticos asociados a ella y su calibración. La segunda parte está destinada al análisis de observabilidad y el diseño de observadores de estado. Los resultados de esta sección son importantes porque dentro de las condiciones de vuelo conflictivas se encuentra el vuelo a punto fijo, una maniobra de vuelo central en muchas misiones de vehículos VToL. Se analizan estas condiciones críticas de vuelo y para ellas se deriva y describe el espacio de soluciones posible conocido como región indistinguible. Los resultados son verificados en simulación dónde se puede apreciar como un estimador de estado falla al intentar realizar su tarea cuando la aeronave está en vuelo a punto fijo. Seguidamente se presenta el diseño de filtros extendidos de Kalman (EKF) que proveen estimaciones del estado con la información limitada de los sensores disponibles. Se discuten conceptos que habitualmente no se presentan en la literatura como cuando actualizar o propagar en el algoritmo de estimación o que sistema de referencia se debe utilizar para representar adecuadamente las variables de estado. Esto lleva a la formulación algorítmica de cuatro filtros discretos basados en eventos, diferentes, pero en esencia equivalentes. Se derivan rutinas de inicialización para los filtros y se comparan los resultados en simulación bajo condiciones favorables de estimación. Con la idea de disminuir el efecto de volar en configuraciones de observabilidad conflictivas, se deriva un filtro alternativo basado en el filtro de Schmidt Kalman (SKF). El filtro propuesto comparte estructura con el EKF, tiene un mejor comportamiento allí dónde le EKF falla y una respuesta similar en el resto de condiciones de vuelo. La última parte de la tesis trata con la estimación de perturbaciones externas. Para ello se deriva un modelo lineal que relaciona fuerzas y momentos con velocidades junto a un modelo de alto orden para las perturbaciones. Se estudia su aplicación a dos modelos para la estimación de perturbaciones ya presentes en la literatura. Además, se proponen dos nuevos filtros de Kalman que se aprovechan de la linealidad del modelo. Se presenta una comparativa basada en la simulación de escenarios ideales así como realistas que demuestra que los filtros KF superan al resto. Esta misma parte de la tesis presenta el diseño genérico de estimadores "quadratic bounded" para sistemas dinámicos lineales cuya incertidumbre se encuentra acotada dentro de elipsoides. Estos estimadores maximizan un compromiso, ajustable por el usuario que contempla la velocidad de convergencia así como el volumen de la solución final que contiene el error de estimación. Se deriva un observador de perturbaciones para plataformas aéreas basado en el modelo de alto orden arriba mencionado. Finalmente, se presenta un análisis del desempeño de estimación en función de los parámetros de diseño del filtro.Postprint (published version

Learning Grasp Strategies Composed of Contact Relative Motions

Of central importance to grasp synthesis algorithms are the assumptions made about the object to be grasped and the sensory information that is available. Many approaches avoid the issue of sensing entirely by assuming that complete information is available. In contrast, this paper proposes an approach to grasp synthesis expressed in terms of units of control that simultaneously change the contact configuration and sense information about the object and the relative manipulator-object pose. These units of control, known as contact relative motions (CRMs), allow the grasp synthesis problem to be recast as an optimal control problem where the goal is to find a strategy for executing CRMs that leads to a grasp in the shortest number of steps. An experiment is described that uses Robonaut, the NASA-JSC space humanoid, to show that CRMs are a viable means of synthesizing grasps. However, because of the limited amount of information that a single CRM can sense, the optimal control problem may be partially observable. This paper proposes expressing the problem as a k-order Markov Decision Process (MDP) and solving it using Reinforcement Learning. This approach is tested in a simulation of a two-contact manipulator that learns to grasp an object. Grasp strategies learned in simulation are tested on the physical Robonaut platform and found to lead to grasp configurations consistently

Control of Cooperating Mobile Manipulators

We describe a framework and control algorithms for coordinating multiple mobile robots with manipulators focusing on tasks that require grasping, manipulation and transporting large and possibly flexible objects without special purpose fixtures. Because each robot has an independent controller and is autonomous, the coordination and synergy are realized through sensing and communication. The robots can cooperatively transport objects and march in a tightly controlled formation, while also having the capability to navigate autonomously. We describe the key aspects of the overall hierarchy and the basic algorithms, with specific applications to our experimental testbed consisting of three robots. We describe results from many experiments that demonstrate the ability of the system to carry flexible boards and large boxes as well as the system’s robustness to alignment and odometry errors