253 research outputs found
From Rolling Over to Walking: Enabling Humanoid Robots to Develop Complex Motor Skills
This paper presents an innovative method for humanoid robots to acquire a
comprehensive set of motor skills through reinforcement learning. The approach
utilizes an achievement-triggered multi-path reward function rooted in
developmental robotics principles, facilitating the robot to learn gross motor
skills typically mastered by human infants within a single training phase. The
proposed method outperforms standard reinforcement learning techniques in
success rates and learning speed within a simulation environment. By leveraging
the principles of self-discovery and exploration integral to infant learning,
this method holds the potential to significantly advance humanoid robot motor
skill acquisition.Comment: 8 pages, 9 figures. Submitted to IEEE Robotics and Automation
Letters. Video available at https://youtu.be/d0RqrW1Ezj
Design a Fall Recovery Strategy for a Wheel-Legged Quadruped Robot Using Stability Feature Space
In this paper, we introduced a conceptual analysis to select stability features when performing predefined and precise motions on robots. By analyzing the different stable poses named features and the possible transitions towards different ones, the introduced concept allows to design more predictable and suitable motions when performing particular tasks. As an example of how the concept can be applied we use it on the fall recovery of the quadruped robot CENTAURO. This robot, which is equipped with a custom hybrid wheel-legged mobility system, have good intrinsic stability as other quadrupeds. However, the characteristics of the rough terrains where it might be deployed require complex maneuvers to cope with possible strong disturbances. To prevent and more importantly recover from falls, realignment of postural responses will not be adequate, and effective recovery procedures should be developed. This paper introduces the details of how the presented conceptual analysis provides and an effective fall recovery routine for CENTAURO based on a state machine. The performance of the proposed approach is evaluated with extensive simulation trials using the dynamic model of the CENTAURO robot showing good effectiveness in recovering the robot after fall on flat and inclined surfaces
Legged locomotion over irregular terrains: State of the art of human and robot performance
Legged robotic technologies have moved out of the lab to operate in real environments, characterized by a wide variety of unpredictable irregularities and disturbances, all this in close proximity with humans. Demonstrating the ability of current robots to move robustly and reliably in these conditions is becoming essential to prove their safe operation. Here, we report an in-depth literature review aimed at verifying the existence of common or agreed protocols and metrics to test the performance of legged system in realistic environments. We primarily focused on three types of robotic technologies, i.e., hexapods, quadrupeds and bipeds. We also included a comprehensive overview on human locomotion studies, being it often considered the gold standard for performance, and one of the most important sources of bioinspiration for legged machines. We discovered that very few papers have rigorously studied robotic locomotion under irregular terrain conditions. On the contrary, numerous studies have addressed this problem on human gait, being nonetheless of highly heterogeneous nature in terms of experimental design. This lack of agreed methodology makes it challenging for the community to properly assess, compare and predict the performance of existing legged systems in real environments. On the one hand, this work provides a library of methods, metrics and experimental protocols, with a critical analysis on the limitations of the current approaches and future promising directions. On the other hand, it demonstrates the existence of an important lack of benchmarks in the literature, and the possibility of bridging different disciplines, e.g., the human and robotic, towards the definition of standardized procedure that will boost not only the scientific development of better bioinspired solutions, but also their market uptake
Locomoção de humanoides robusta e versátil baseada em controlo analĂtico e fĂsica residual
Humanoid robots are made to resemble humans but their locomotion
abilities are far from ours in terms of agility and versatility. When humans
walk on complex terrains or face external disturbances, they
combine a set of strategies, unconsciously and efficiently, to regain
stability. This thesis tackles the problem of developing a robust omnidirectional
walking framework, which is able to generate versatile
and agile locomotion on complex terrains. We designed and developed
model-based and model-free walk engines and formulated the
controllers using different approaches including classical and optimal
control schemes and validated their performance through simulations
and experiments. These frameworks have hierarchical structures that
are composed of several layers. These layers are composed of several
modules that are connected together to fade the complexity and
increase the flexibility of the proposed frameworks. Additionally, they
can be easily and quickly deployed on different platforms.
Besides, we believe that using machine learning on top of analytical approaches
is a key to open doors for humanoid robots to step out of laboratories.
We proposed a tight coupling between analytical control and
deep reinforcement learning. We augmented our analytical controller
with reinforcement learning modules to learn how to regulate the walk
engine parameters (planners and controllers) adaptively and generate
residuals to adjust the robot’s target joint positions (residual physics).
The effectiveness of the proposed frameworks was demonstrated and
evaluated across a set of challenging simulation scenarios. The robot
was able to generalize what it learned in one scenario, by displaying
human-like locomotion skills in unforeseen circumstances, even in the
presence of noise and external pushes.Os robĂ´s humanoides sĂŁo feitos para se parecerem com humanos,
mas suas habilidades de locomoção estão longe das nossas em termos
de agilidade e versatilidade. Quando os humanos caminham em
terrenos complexos ou enfrentam distĂşrbios externos combinam diferentes
estratégias, de forma inconsciente e eficiente, para recuperar a
estabilidade. Esta tese aborda o problema de desenvolver um sistema
robusto para andar de forma omnidirecional, capaz de gerar uma locomoção
para robôs humanoides versátil e ágil em terrenos complexos.
Projetámos e desenvolvemos motores de locomoção sem modelos e
baseados em modelos. Formulámos os controladores usando diferentes
abordagens, incluindo esquemas de controlo clássicos e ideais,
e validámos o seu desempenho por meio de simulações e experiências
reais. Estes frameworks têm estruturas hierárquicas compostas por
várias camadas. Essas camadas são compostas por vários módulos
que sĂŁo conectados entre si para diminuir a complexidade e aumentar
a flexibilidade dos frameworks propostos. Adicionalmente, o sistema
pode ser implementado em diferentes plataformas de forma fácil.
Acreditamos que o uso de aprendizagem automática sobre abordagens
analĂticas Ă© a chave para abrir as portas para robĂ´s humanoides
saĂrem dos laboratĂłrios. Propusemos um forte acoplamento entre controlo
analĂtico e aprendizagem profunda por reforço. Expandimos o
nosso controlador analĂtico com mĂłdulos de aprendizagem por reforço
para aprender como regular os parâmetros do motor de caminhada
(planeadores e controladores) de forma adaptativa e gerar resĂduos
para ajustar as posições das juntas alvo do robĂ´ (fĂsica residual). A
eficácia das estruturas propostas foi demonstrada e avaliada em um
conjunto de cenários de simulação desafiadores. O robô foi capaz de
generalizar o que aprendeu em um cenário, exibindo habilidades de
locomoção humanas em circunstâncias imprevistas, mesmo na presença
de ruĂdo e impulsos externos.Programa Doutoral em Informátic
Off-Policy Temporal Difference Learning For Robotics And Autonomous Systems
Reinforcement learning (RL) is a rapidly advancing field with implications in autonomous vehicles, medicine, finance, along with several other applications. Particularly, off-policy temporal difference (TD) learning, a specific type of RL technique, has been widely used in a variety of autonomous tasks. However, there remain significant challenges that must be overcome before it can be successfully applied to various real-world applications. In this thesis, we specifically address several major challenges in off-policy TD learning.
In the first part of the thesis, we introduce an efficient method of learning complex stand-up motion of humanoid robots by Q-learning. Standing up after falling is an essential ability for humanoid robots yet it is difficult to learn flexible stand-up motions for various fallen positions due to the complexity of the task. We reduce sample complexity of learning by applying a clustering method and utilizing the bilateral symmetric feature of humanoid robots. The learned policy is demonstrated in both simulation and on a physical robot.
The greedy update of Q-learning, however, often causes overoptimism and instability. In the second part of the thesis, we propose a novel Bayesian approach to Q-learning, called ADFQ, which improves the greedy update issues by providing a principled way of updating Q-values based on uncertainty of Q-belief distributions. The algorithm converges to Q-learning as the uncertainty approaches zero, and its efficient computational complexity enables the algorithm to be extended with a neural network. Both ADFQ and its neural network extension outperform their comparing algorithms by improving the estimation bias and converging faster to optimal Q-values.
In the last part of the thesis, we apply off-policy TD methods to solve the active information acquisition problem where an autonomous agent is tasked with acquiring information about targets of interests. Off-policy TD learning provides solutions for classical challenges in this problem -- system model dependence and the difficulty of computing information-theoretic cost functions for a long planning horizon. In particular, we introduce a method of learning a unified policy for in-sight tracking, navigation, and exploration. The policy shows robust behavior for tracking agile and anomalous targets with a partially known target model
System Identification of Bipedal Locomotion in Robots and Humans
The ability to perform a healthy walking gait can be altered in numerous cases due to gait disorder related pathologies. The latter could lead to partial or complete mobility loss, which affects the patients’ quality of life. Wearable exoskeletons and active prosthetics have been considered as a key component to remedy this mobility loss. The control of such devices knows numerous challenges that are yet to be addressed. As opposed to fixed trajectories control, real-time adaptive reference generation control is likely to provide the wearer with more intent control over the powered device. We propose a novel gait pattern generator for the control of such devices, taking advantage of the inter-joint coordination in the human gait. Our proposed method puts the user in the control loop as it maps the motion of healthy limbs to that of the affected one. To design such control strategy, it is critical to understand the dynamics behind bipedal walking. We begin by studying the simple compass gait walker. We examine the well-known Virtual Constraints method of controlling bipedal robots in the image of the compass gait. In addition, we provide both the mechanical and control design of an affordable research platform for bipedal dynamic walking. We then extend the concept of virtual constraints to human locomotion, where we investigate the accuracy of predicting lower limb joints angular position and velocity from the motion of the other limbs. Data from nine healthy subjects performing specific locomotion tasks were collected and are made available online. A successful prediction of the hip, knee, and ankle joints was achieved in different scenarios. It was also found that the motion of the cane alone has sufficient information to help predict good trajectories for the lower limb in stairs ascent. Better estimates were obtained using additional information from arm joints. We also explored the prediction of knee and ankle trajectories from the motion of the hip joints
Integrating Vision and Physical Interaction for Discovery, Segmentation and Grasping of Unknown Objects
In dieser Arbeit werden Verfahren der Bildverarbeitung und die Fähigkeit
humanoider Roboter, mit ihrer Umgebung physisch zu interagieren, in engem
Zusammenspiel eingesetzt, um unbekannte Objekte zu identifizieren, sie vom
Hintergrund und anderen Objekten zu trennen, und letztendlich zu greifen.
Im Verlauf dieser interaktiven Exploration werden auĂźerdem Eigenschaften
des Objektes wie etwa sein Aussehen und seine Form ermittelt
Trial-and-Error Learning of Repulsors for Humanoid QP-based Whole-Body Control
International audienceWhole body controllers based on quadratic programming allow humanoid robots to achieve complex motions. However, they rely on the assumption that the model perfectly captures the dynamics of the robot and its environment, whereas even the most accurate models are never perfect. In this paper, we introduce a trial-and-error learning algorithm that allows whole-body controllers to operate in spite of inaccurate models, without needing to update these models. The main idea is to encourage the controller to perform the task differently after each trial by introducing repulsors in the quadratic program cost function. We demonstrate our algorithm on (1) a simple 2D case and (2) a simulated iCub robot for which the model used by the controller and the one used in simulation do not match
Enabling Human-Robot Collaboration via Holistic Human Perception and Partner-Aware Control
As robotic technology advances, the barriers to the coexistence of humans and robots are slowly coming down. Application domains like elderly care, collaborative manufacturing, collaborative manipulation, etc., are considered the need of the hour, and progress in robotics holds the potential to address many societal challenges. The future socio-technical systems constitute of blended workforce with a symbiotic relationship between human and robot partners working collaboratively. This thesis attempts to address some of the research challenges in enabling human-robot collaboration. In particular, the challenge of a holistic perception of a human partner to continuously communicate his intentions and needs in real-time to a robot partner is crucial for the successful realization of a collaborative task. Towards that end, we present a holistic human perception framework for real-time monitoring of whole-body human motion and dynamics. On the other hand, the challenge of leveraging assistance from a human partner will lead to improved human-robot collaboration. In this direction, we attempt at methodically defining what constitutes assistance from a human partner and propose partner-aware robot control strategies to endow robots with the capacity to meaningfully engage in a collaborative task
- …