Application of Hidden Markov Model to locate soccer robots

© 2015 Technical Committee on Control Theory, Chinese Association of Automation. This paper adopts a Hidden Markov Model as a basis for predicting the probabilities in location of soccer robot's trajectories, develops the corresponding algorithms, and then demonstrates the simplicity of the procedure with simulations. The purpose of the initial presentation is to establish a proper platform for the future comprehensive studies of using Hidden Markov Models to assemble critical observations with uncertainties or random measurement errors in stochastic system modelling and control

ROBOT LEARNING OF OBJECT MANIPULATION TASK ACTIONS FROM HUMAN DEMONSTRATIONS

Robot learning from demonstration is a method which enables robots to learn in a similar way as humans. In this paper, a framework that enables robots to learn from multiple human demonstrations via kinesthetic teaching is presented. The subject of learning is a high-level sequence of actions, as well as the low-level trajectories necessary to be followed by the robot to perform the object manipulation task. The multiple human demonstrations are recorded and only the most similar demonstrations are selected for robot learning. The high-level learning module identifies the sequence of actions of the demonstrated task. Using Dynamic Time Warping (DTW) and Gaussian Mixture Model (GMM), the model of demonstrated trajectories is learned. The learned trajectory is generated by Gaussian mixture regression (GMR) from the learned Gaussian mixture model.  In online working phase, the sequence of actions is identified and experimental results show that the robot performs the learned task successfully

How can a recurrent neurodynamic predictive coding model cope with fluctuation in temporal patterns? Robotic experiments on imitative interaction

The current paper examines how a recurrent neural network (RNN) model using a dynamic predictive coding scheme can cope with fluctuations in temporal patterns through generalization in learning. The conjecture driving this present inquiry is that a RNN model with multiple timescales (MTRNN) learns by extracting patterns of change from observed temporal patterns, developing an internal dynamic structure such that variance in initial internal states account for modulations in corresponding observed patterns. We trained a MTRNN with low-dimensional temporal patterns, and assessed performance on an imitation task employing these patterns. Analysis reveals that imitating fluctuated patterns consists in inferring optimal internal states by error regression. The model was then tested through humanoid robotic experiments requiring imitative interaction with human subjects. Results show that spontaneous and lively interaction can be achieved as the model successfully copes with fluctuations naturally occurring in human movement patterns

Robot Learning Dual-Arm Manipulation Tasks by Trial-and-Error and Multiple Human Demonstrations

In robotics, there is a need of an interactive and expedite learning method as experience is expensive. In this research, we propose two different methods to make a humanoid robot learn manipulation tasks: Learning by trial-and-error, and Learning from demonstrations. Just like the way a child learns a new task assigned to him by trying all possible alternatives and further learning from his mistakes, the robot learns in the same manner in learning by trial-and error. We used Q-learning algorithm, in which the robot tries all the possible ways to do a task and creates a matrix that consists of Q-values based on the rewards it received for the actions performed. Using this method, the robot was made to learn dance moves based on a music track. Robot Learning from Demonstrations (RLfD) enable a human user to add new capabilities to a robot in an intuitive manner without explicitly reprogramming it. In this method, the robot learns skill from demonstrations performed by a human teacher. The robot extracts features from each demonstration called as key-points and learns a model of the demonstrated task or trajectory using Hidden Markov Model (HMM). The learned model is further used to produce a generalized trajectory. In the end, we discuss the differences between two developed systems and make conclusions based on the experiments performed

Trajectory learning for robot programming by demonstration using hidden markov model and dynamic time warping

The main objective of this paper is to develop an efficient method for learning and reproduction of complex trajectories for robot programming by demonstration. Encoding of the demonstrated trajectories is performed with hidden Markov model, and generation of a generalized trajectory is achieved by using the concept of key points. Identification of the key points is based on significant changes in position and velocity in the demonstrated trajectories. The resulting sequences of trajectory key points are temporally aligned using the multidimensional dynamic time warping algorithm, and a generalized trajectory is obtained by smoothing spline interpolation of the clustered key points. The principal advantage of our proposed approach is utilization of the trajectory key points from all demonstrations for generation of a generalized trajectory. In addition, variability of the key points' clusters across the demonstrated set is employed for assigning weighting coefficients, resulting in a generalization procedure which accounts for the relevance of reproduction of different parts of the trajectories. The approach is verified experimentally for trajectories with two different levels of complexity. \ua9 2012 IEEE.Peer reviewed: YesNRC publication: Ye

Robot Learning from Human Demonstration: Interpretation, Adaptation, and Interaction

Robot Learning from Demonstration (LfD) is a research area that focuses on how robots can learn new skills by observing how people perform various activities. As humans, we have a remarkable ability to imitate other human’s behaviors and adapt to new situations. Endowing robots with these critical capabilities is a significant but very challenging problem considering the complexity and variation of human activities in highly dynamic environments. This research focuses on how robots can learn new skills by interpreting human activities, adapting the learned skills to new situations, and naturally interacting with humans. This dissertation begins with a discussion of challenges in each of these three problems. A new unified representation approach is introduced to enable robots to simultaneously interpret the high-level semantic meanings and generalize the low-level trajectories of a broad range of human activities. An adaptive framework based on feature space decomposition is then presented for robots to not only reproduce skills, but also autonomously and efficiently adjust the learned skills to new environments that are significantly different from demonstrations. To achieve natural Human Robot Interaction (HRI), this dissertation presents a Recurrent Neural Network based deep perceptual control approach, which is capable of integrating multi-modal perception sequences with actions for robots to interact with humans in long-term tasks. Overall, by combining the above approaches, an autonomous system is created for robots to acquire important skills that can be applied to human-centered applications. Finally, this dissertation concludes with a discussion of future directions that could accelerate the upcoming technological revolution of robot learning from human demonstration

Trajectory planning and control for robot manipulations

Comme les robots effectuent de plus en plus de tâches en interaction avec l'homme ou dans un environnement humain, ils doivent assurer la sécurité et le confort des hommes. Dans ce contexte, le robot doit adapter son comportement et agir en fonction des évolutions de l'environnement et des activités humaines. Les robots développés sur la base de l'apprentissage ou d'un planificateur de mouvement ne sont pas en mesure de réagir assez rapidement, c'est pourquoi nous proposons d'introduire un contrôleur de trajectoire intermédiaire dans l'architecture logicielle entre le contrôleur bas niveau et le planificateur de plus haut niveau. Le contrôleur de trajectoire que nous proposons est basé sur le concept de générateur de trajectoire en ligne (OTG), il permet de calculer des trajectoires en temps réel et facilite la communication entre les différents éléments, en particulier le planificateur de chemin, le générateur de trajectoire, le détecteur de collision et le contrôleur. Pour éviter de replanifier toute une trajectoire en réaction à un changement induit par un humain, notre contrôleur autorise la déformation locale de la trajectoire et la modification de la loi d'évolution pour accélérer ou décélérer le mouvement. Le contrôleur de trajectoire peut également commuter de la trajectoire initiale vers une nouvelle trajectoire. Les fonctions polynomiales cubiques que nous utilisons pour décrire les trajectoires fournissent des mouvements souples et de la flexibilité sans nécessiter de calculs complexes. De plus, les algorithmes de lissage que nous proposons permettent de produire des mouvements esthétiques ressemblants à ceux des humains. Ce travail, mené dans le cadre du projet ANR ICARO, a été intégré et validé avec les robots KUKA LWR de la plate-forme robotique du LAAS-CNRS.In order to perform a large variety of tasks in interaction with human or in human environments, a robot needs to guarantee safety and comfort for humans. In this context, the robot shall adapt its behavior and react to the environment changes and human activities. The robots based on learning or motion planning are not able to adapt fast enough, so we propose to use a trajectory controller as an intermediate control layer in the software structure. This intermediate layer exchanges information with the low level controller and the high level planner. The proposed trajectory controller, based on the concept of Online Trajectory Generation (OTG), allows real time computation of trajectories and easy communication with the different components, including path planner, trajectory generator, collision checker and controller. To avoid the replan of an entire trajectory when reacting to a human behaviour change, the controller must allow deforming locally a trajectory or accelerate/decelerate by modifying the time function. The trajectory controller must also accept to switch from an initial trajectory to a new trajectory to follow. Cubic polynomial functions are used to describe trajectories, they provide smoothness, flexibility and computational simplicity. Moreover, to satisfy the objective of aesthetics, smoothing algorithm are proposed to produce human-like motions. This work, conducted as part of the ANR project ICARO, has been integrated and validated on the KUKA LWR robot platform of LAAS-CNRS

Robots that Learn and Plan — Unifying Robot Learning and Motion Planning for Generalized Task Execution

Robots have the potential to assist people with a variety of everyday tasks, but to achieve that potential robots require software capable of planning and executing motions in cluttered environments. To address this, over the past few decades, roboticists have developed numerous methods for planning motions to avoid obstacles with increasingly stronger guarantees, from probabilistic completeness to asymptotic optimality. Some of these methods have even considered the types of constraints that must be satisfied to perform useful tasks, but these constraints must generally be manually specified. In recent years, there has been a resurgence of methods for automatic learning of tasks from human-provided demonstrations. Unfortunately, these two fields, task learning and motion planning, have evolved largely separate from one another, and the learned models are often not usable by motion planners. In this thesis, we aim to bridge the gap between robot task learning and motion planning by employing a learned task model that can subsequently be leveraged by an asymptotically-optimal motion planner to autonomously execute the task. First, we show that application of a motion planner enables task performance while avoiding novel obstacles and extend this to dynamic environments by replanning at reactive rates. Second, we generalize the method to accommodate time-invariant model parameters, allowing more information to be gleaned from the demonstrations. Third, we describe a more principled approach to temporal registration for such learning methods that mirrors the ultimate integration with a motion planner and often reduces the number of demonstrations required. Finally, we extend this framework to the domain of mobile manipulation. We empirically evaluate each of these contributions on multiple household tasks using the Aldebaran Nao, Rethink Robotics Baxter, and Fetch mobile manipulator robots to show that these approaches improve task execution success rates and reduce the amount of human-provided information required.Doctor of Philosoph