Non-Decoupled Locomotion and Manipulation Planning for Low-Dimensional Systems

International audienceWe demonstrate the possibility of solving planning problems by inter-leaving locomotion and manipulation in a non-decoupled way. We choose three low-dimensional minimalistic robotic systems and use them to illustrate our paradigm: a basic one-legged locomotor, a two-link manipulator with a manipulated object, and a simultaneous locomotion-and-manipulation system. Using existing motion planning and control methods initially designed for either locomotion or manipulation tasks, we see how they apply to both our locomotion-only and manipulation-only systems through parallel derivations, and extend them to the simultaneous locomotion-and-manipulation system. Motion planning is solved for these three systems using two different methods : (i) a geometric path-planning-based one, and (ii) a kinematic control-theoretic-based one. Motion control is then derived by dynamically realizing the geometric paths or kinematic trajectories under the Couloumb friction model using torques as control inputs. All three methods apply successfully to all three systems, showing that the non-decoupled planning is possible

Adaptive Whole-Body Manipulation in Human-to-Humanoid Multi-Contact Motion Retargeting

Submitted to Humanoids 2017International audienceWe propose a multi-robot quadratic program (QP) controller for retargeting of a human's multi-contact loco-manipulation motions to a humanoid robot. Using this framework , the robot can track complex motions and automatically adapt to objects in the environment that have different physical properties from those that were used to provide the human's reference motion. The whole-body multi-contact manipulation problem is formulated as a multi-robot QP, which optimizes over the combined dynamics of the robot and any manipulated objects. The multi-robot QP maintains a dynamic partition of the robot's tracking links into fixed support contact, manipulation contact, and contact-free tracking links, which are re-partitioned and re-instantiated as constraints in the multi-robot QP every time a contact event occurs in the human motion. We present various experiments (bag retrieval, door opening, box lifting) using human motion data from an Xsens inertial motion capture system. We show in full-body dynamics simulation that the robot is able to perform difficult single-stance motions as well as multi-contact-stance motions (including hand supports), while adapting to objects of varying inertial properties

On Weight-Prioritized Multi-Task Control of Humanoid Robots

International audienceWe propose a formal analysis with some theoretical properties of weight-prioritized multi-task inverse-dynamics-like control of humanoid robots, being a case of redundant " ma-nipulators " with a non-actuated free-floating base and multiple unilateral frictional contacts with the environment. The controller builds on a weighted sum scalarization of a multiobjective optimization problem under equality and inequality constraints, which appears as a straightforward solution to account for state and control input viability constraints characteristic of humanoid robots that were usually absent from early existing pseudo-inverse and null-space projection-based prioritized multi-task approaches. We argue that our formulation is indeed well founded and justified from a theoretical standpoint, and we propose an analysis of some stability properties of the approach: Lyapunov stability is demonstrated for the closed-loop dynamical system that we analytically derive in the unconstrained multiob-jective optimization case. Stability in terms of solution existence, uniqueness, continuity, and robustness to perturbations, is then formally demonstrated for the constrained quadratic program

QP-based Adaptive-Gains Compliance Control in Humanoid Falls

International audienceWe address the problem of humanoid falling with a decoupled strategy consisting of a pre-impact and a postimpact stage. In the pre-impact stage, geometrical reasoning allows the robot to choose appropriate impact points in the surrounding environment and to adopt a posture to reach them while avoiding impact-singularities and preparing for the postimpact. The surrounding environment can be unstructured and may contain cluttered obstacles. The post-impact stage uses a quadratic program controller that adapts on-line the joint proportional-derivative (PD) gains to make the robot compliant-to absorb impact and post-impact dynamics, which lowers possible damage risks. This is done by a new approach incorporating the stiffness and damping gains directly as decision variables in the QP along with the usually-considered variables of joint accelerations and contact forces. Constraints of the QP prevent the motors from reaching their torque limits during the fall. Several experiments on the humanoid robot HRP-4 in a full-dynamics simulator are presented and discussed

DreamPaint: Few-Shot Inpainting of E-Commerce Items for Virtual Try-On without 3D Modeling

We introduce DreamPaint, a framework to intelligently inpaint any e-commerce product on any user-provided context image. The context image can be, for example, the user's own image for virtual try-on of clothes from the e-commerce catalog on themselves, the user's room image for virtual try-on of a piece of furniture from the e-commerce catalog in their room, etc. As opposed to previous augmented-reality (AR)-based virtual try-on methods, DreamPaint does not use, nor does it require, 3D modeling of neither the e-commerce product nor the user context. Instead, it directly uses 2D images of the product as available in product catalog database, and a 2D picture of the context, for example taken from the user's phone camera. The method relies on few-shot fine tuning a pre-trained diffusion model with the masked latents (e.g., Masked DreamBooth) of the catalog images per item, whose weights are then loaded on a pre-trained inpainting module that is capable of preserving the characteristics of the context image. DreamPaint allows to preserve both the product image and the context (environment/user) image without requiring text guidance to describe the missing part (product/context). DreamPaint also allows to intelligently infer the best 3D angle of the product to place at the desired location on the user context, even if that angle was previously unseen in the product's reference 2D images. We compare our results against both text-guided and image-guided inpainting modules and show that DreamPaint yields superior performance in both subjective human study and quantitative metrics

Trial-and-Error Learning of Repulsors for Humanoid QP-based Whole-Body Control

International audienceWhole body controllers based on quadratic programming allow humanoid robots to achieve complex motions. However, they rely on the assumption that the model perfectly captures the dynamics of the robot and its environment, whereas even the most accurate models are never perfect. In this paper, we introduce a trial-and-error learning algorithm that allows whole-body controllers to operate in spite of inaccurate models, without needing to update these models. The main idea is to encourage the controller to perform the task differently after each trial by introducing repulsors in the quadratic program cost function. We demonstrate our algorithm on (1) a simple 2D case and (2) a simulated iCub robot for which the model used by the controller and the one used in simulation do not match

On Autonomous Behaviour of Humanoid Robots : Contact Planning for Locomotion and Manipulation

Nous proposons une approche de planification unifiée pour robots humanoïdes réalisant des tâches de locomotion et de manipulation nécessitant une dextérité propre aux systèmes anthropomorphes. Ces tâches sont basées sur des transitions de contacts ; contacts entre les extrémités des membres locomoteurs et l'environnement dans le cas du problème de locomotion par exemple, ou entre les extrémités de l'organe préhensible effecteur et l'objet manipulé dans le cas du problème de manipulation. Nous planifions ces transitions de contacts pour des systèmes abstraits constitués d'autant de robots, d'objets, et de supports dans l'environnement que désiré/nécessaire pour la modélisation du problème. Cette approche permet de s'affranchir de la distinction de nature entre tâches de locomotion et de manipulation et s'étend à une variété d'autres problèmes tels que la coopération entre plusieurs agents. Nous introduisons notre paradigme de planification non-découplée de locomotion et de manipulation en exhibant la stratification induite dans l'espace des configurations de systèmes simplifiés pour lesquels nous résolvons analytiquement le problème en comparant des méthodes de planification géométrique, non-holonome, et dynamique. Nous présentons ensuite l'algorithme de planification de contacts basé sur une recherche best-first. Cet algorithme fait appel à un solveur de cinématique inverse qui prend en compte des configurations de contacts générales dans l'espace pouvant être établis entre robots, objets, et environnement dans toutes les combinaisons possibles, le tout sous contraintes d'équilibre statique et de respect des limitations mécaniques des robots. La génération de mouvement respectant l'équation de dynamique Lagrangienne est obtenue par une formulation en programme quadratique. Enfin nous envisageons une extension à des supports de contact déformables en considérant des comportements linéaires-élastiques résolus par éléments finis.We propose a unified planning approach for autonomous humanoid robots that perform dexterous locomotion and manipulation tasks. These tasks are based on contact transitions; for instance between the locomotion limbs of the robot and the environment, or between the manipulation end-effector of the robot and the manipulated object. We plan these contact transitions for general abstract systems made of arbitrary numbers of robots, manipulated objects, and environment supports. This approach allows us to erase distinction between the locomotion and manipulation nature of the tasks and to extend the method to various other planning problems such as collaborative manipulation and locomotion between multiple agents. We introduce our non-decoupled locomotion-and-manipulation planning paradigm by exhibiting the induced stratification of the configuration space of example simplified systems for which we analytically solve the problem comparing geometric path planning, kinematic non-holonomic planning, and dynamic trajectory planning methods. We then present the contact planning algorithm based on best-first search. The algorithm relies on an inverse kinematics solver that handles general robot-robot, robot-object, robot-environment, object-environment, non-horizontal, non-coplanar, friction-based, multi-contact configurations, under static equilibrium and physical limitation constraints. The continuous dynamics-consistent motion is generated in the locomotion case using a quadratic programming formulation. We finally envision the extension to deformable environment contact support by considering linear elasticity behaviours solved using the finite element method

De l'Autonomie des Robots Humanoïdes : Planification de Contacts pour Mouvements de Locomotion et Tâches de Manipulation

We propose a unified planning approach for autonomous humanoid robots that perform dexterous locomotion and manipulation tasks. These tasks are based on contact transitions; for instance between the locomotion limbs of the robot and the environment, or between the manipulation end-effector of the robot and the manipulated object. We plan these contact transitions for general abstract systems made of arbitrary numbers of robots, manipulated objects, and environment supports. This approach allows us to erase distinction between the locomotion and manipulation nature of the tasks and to extend the method to various other planning problems such as collaborative manipulation and locomotion between multiple agents. We introduce our non-decoupled locomotion-and-manipulation planning paradigm by exhibiting the induced stratification of the configuration space of example simplified systems for which we analytically solve the problem comparing geometric path planning, kinematic non-holonomic planning, and dynamic trajectory planning methods. We then present the contact planning algorithm based on best-first search. The algorithm relies on an inverse kinematics solver that handles general robot-robot, robot-object, robot-environment, object-environment, non-horizontal, non-coplanar, friction-based, multi-contact configurations, under static equilibrium and physical limitation constraints. The continuous dynamics-consistent motion is generated in the locomotion case using a quadratic programming formulation. We finally envision the extension to deformable environment contact support by considering linear elasticity behaviours solved using the finite element method.Nous proposons une approche de planification unifiée pour robots humanoïdes réalisant des tâches de locomotion et de manipulation nécessitant une dextérité propre aux systèmes anthropomorphes. Ces tâches sont basées sur des transitions de contacts ; contacts entre les extrémités des membres locomoteurs et l'environnement dans le cas du problème de locomotion par exemple, ou entre les extrémités de l'organe préhensible effecteur et l'objet manipulé dans le cas du problème de manipulation. Nous planifions ces transitions de contacts pour des systèmes abstraits constitués d'autant de robots, d'objets, et de supports dans l'environnement que désiré/nécessaire pour la modélisation du problème. Cette approche permet de s'affranchir de la distinction de nature entre tâches de locomotion et de manipulation et s'étend à une variété d'autres problèmes tels que la coopération entre plusieurs agents. Nous introduisons notre paradigme de planification non-découplée de locomotion et de manipulation en exhibant la stratification induite dans l'espace des configurations de systèmes simplifiés pour lesquels nous résolvons analytiquement le problème en comparant des méthodes de planification géométrique, non-holonome, et dynamique. Nous présentons ensuite l'algorithme de planification de contacts basé sur une recherche best-first. Cet algorithme fait appel à un solveur de cinématique inverse qui prend en compte des configurations de contacts générales dans l'espace pouvant être établis entre robots, objets, et environnement dans toutes les combinaisons possibles, le tout sous contraintes d'équilibre statique et de respect des limitations mécaniques des robots. La génération de mouvement respectant l'équation de dynamique Lagrangienne est obtenue par une formulation en programme quadratique. Enfin nous envisageons une extension à des supports de contact déformables en considérant des comportements linéaires-élastiques résolus par éléments finis

On the dynamics modeling of free-floating-base articulated mechanisms and applications to humanoid whole-body dynamics and control

International audienceWe propose in this paper a general analytic scheme based on Gauss principle of least constraint for the derivation of the Lagrangian dynamics equation of motion of arbitrarily parameterized free-floating-base articulated mechanisms. The free-floating base of the mechanism is a non-actuated rigid object evolving in the 6D Lie group SE(3), the SO(3) component of which can be parameterized using arbitrary coordinate charts with equality constraints, for instance unit quaternions (also known as Euler parameters). This class of systems includes humanoid robots, and the presented formalism is particularly suitable for the whole-body dynamics modeling and control problem of such humanoid systems. Example motions of humanoid in arbitrary contact states with the environment demonstrate the originality of the approach