Structured Prediction for CRiSP Inverse Kinematics Learning with Misspecified Robot Models

With the recent advances in machine learning, problems that traditionally would require accurate modeling to be solved analytically can now be successfully approached with data-driven strategies. Among these, computing the inverse kinematics of a redundant robot arm poses a significant challenge due to the non-linear structure of the robot, the hard joint constraints and the non-invertible kinematics map. Moreover, most learning algorithms consider a completely data-driven approach, while often useful information on the structure of the robot is available and should be positively exploited. In this work, we present a simple, yet effective, approach for learning the inverse kinematics. We introduce a structured prediction algorithm that combines a data-driven strategy with the model provided by a forward kinematics function -- even when this function is misspecified -- to accurately solve the problem. The proposed approach ensures that predicted joint configurations are well within the robot's constraints. We also provide statistical guarantees on the generalization properties of our estimator as well as an empirical evaluation of its performance on trajectory reconstruction tasks.Comment: Accepted for publication in IEEE Robotics and Automation Letters (2021) and presentation at IEEE International Conference on Robotics and Automation (2021) Updated funding informatio

Reconfigurable Computing Applied to Latency Reduction for the Tactile Internet

Tactile internet applications allow robotic devices to be remotely controlled over a communication medium with an unnoticeable time delay. In a bilateral communication, the acceptable round trip latency is usually in the order of 1ms up to 10ms depending on the application requirements. It is estimated that 70% of the total latency is generated by the communication network, and the remaining 30% is produced by master and slave devices. Thus, this paper aims to propose a strategy to reduce 30% of the total latency that is produced by such devices. The strategy is to apply reconfigurable computation using FPGAs to minimize the execution time of device-associated algorithms. With this in mind, this work presents a hardware reference model for modules that implement nonlinear positioning and force calculations as well as a tactile system formed by two robotic manipulators. In addition to presenting the implementation details, simulations and experimental tests are performed in order to validate the proposed model. Results associated with the FPGA sampling rate, throughput, latency, and post-synthesis occupancy area are analyzed.Comment: 20 pages, 32 Figure

Effizientes und stabiles online Lernen für "Developmental Robots"

Recent progress in robotics and cognitive science has inspired a new generation of more versatile robots, so-called developmental robots. Many learning approaches for these robots are inspired by developmental processes and learning mechanisms observed in children. It is widely accepted that developmental robots must autonomously develop, acquire their skills, and cope with unforeseen challenges in unbounded environments through lifelong learning. Continuous online adaptation and intrinsically motivated learning are thus essential capabilities for these robots. However, the high sample-complexity of online learning and intrinsic motivation methods impedes the efficiency and practical feasibility of these methods for lifelong learning. Consequently, the majority of previous work has been demonstrated only in simulation. This thesis devises new methods and learning schemes to mitigate this problem and to permit direct online training on physical robots. A novel intrinsic motivation method is developed to drive the robot’s exploration to efficiently select what to learn. This method combines new knowledge-based and competence-based signals to increase sample-efficiency and to enable lifelong learning. While developmental robots typically acquire their skills through self-exploration, their autonomous development could be accelerated by additionally learning from humans. Yet there is hardly any research to integrate intrinsic motivation with learning from a teacher. The thesis therefore establishes a new learning scheme to integrate intrinsic motivation with learning from observation. The underlying exploration mechanism in the proposed learning schemes relies on Goal Babbling as a goal-directed method for learning direct inverse robot models online, from scratch, and in a learning while behaving fashion. Online learning of multiple solutions for redundant robots with this framework was missing. This thesis devises an incremental online associative network to enable simultaneous exploration and solution consolidation and establishes a new technique to stabilize the learning system. The proposed methods and learning schemes are demonstrated for acquiring reaching skills. Their efficiency, stability, and applicability are benchmarked in simulation and demonstrated on a physical 7-DoF Baxter robot arm.Jüngste Entwicklungen in der Robotik und den Kognitionswissenschaften haben zu einer Generation von vielseitigen Robotern geführt, die als ”Developmental Robots” bezeichnet werden. Lernverfahren für diese Roboter sind inspiriert von Lernmechanismen, die bei Kindern beobachtet wurden. ”Developmental Robots” müssen autonom Fertigkeiten erwerben und unvorhergesehene Herausforderungen in uneingeschränkten Umgebungen durch lebenslanges Lernen meistern. Kontinuierliches Anpassen und Lernen durch intrinsische Motivation sind daher wichtige Eigenschaften. Allerdings schränkt der hohe Aufwand beim Generieren von Datenpunkten die praktische Nutzbarkeit solcher Verfahren ein. Daher wurde ein Großteil nur in Simulationen demonstriert. In dieser Arbeit werden daher neue Methoden konzipiert, um dieses Problem zu meistern und ein direktes Online-Training auf realen Robotern zu ermöglichen. Dazu wird eine neue intrinsisch motivierte Methode entwickelt, die während der Umgebungsexploration effizient auswählt, was gelernt wird. Sie kombiniert neue wissens- und kompetenzbasierte Signale, um die Sampling-Effizienz zu steigern und lebenslanges Lernen zu ermöglichen. Während ”Developmental Robots” Fertigkeiten durch Selbstexploration erwerben, kann ihre Entwicklung durch Lernen durch Beobachten beschleunigt werden. Dennoch gibt es kaum Arbeiten, die intrinsische Motivation mit Lernen von interagierenden Lehrern verbinden. Die vorliegende Arbeit entwickelt ein neues Lernschema, das diese Verbindung schafft. Der in den vorgeschlagenen Lernmethoden genutzte Explorationsmechanismus beruht auf Goal Babbling, einer zielgerichteten Methode zum Lernen inverser Modelle, die online-fähig ist, kein Vorwissen benötigt und Lernen während der Ausführung von Bewegungen ermöglicht. Das Online-Lernen mehrerer Lösungen inverser Modelle redundanter Roboter mit Goal Babbling wurde bisher nicht erforscht. In dieser Arbeit wird dazu ein inkrementell lernendes, assoziatives neuronales Netz entwickelt und eine Methode konzipiert, die es stabilisiert. Das Netz ermöglicht deren gleichzeitige Exploration und Konsolidierung. Die vorgeschlagenen Verfahren werden für das Greifen nach Objekten demonstriert. Ihre Effizienz, Stabilität und Anwendbarkeit werden simulativ verglichen und mit einem Roboter mit sieben Gelenken demonstriert

From visuomotor control to latent space planning for robot manipulation

Deep visuomotor control is emerging as an active research area for robot manipulation. Recent advances in learning sensory and motor systems in an end-to-end manner have achieved remarkable performance across a range of complex tasks. Nevertheless, a few limitations restrict visuomotor control from being more widely adopted as the de facto choice when facing a manipulation task on a real robotic platform. First, imitation learning-based visuomotor control approaches tend to suffer from the inability to recover from an out-of-distribution state caused by compounding errors. Second, the lack of versatility in task definition limits skill generalisability. Finally, the training data acquisition process and domain transfer are often impractical. In this thesis, individual solutions are proposed to address each of these issues. In the first part, we find policy uncertainty to be an effective indicator of potential failure cases, in which the robot is stuck in out-of-distribution states. On this basis, we introduce a novel uncertainty-based approach to detect potential failure cases and a recovery strategy based on action-conditioned uncertainty predictions. Then, we propose to employ visual dynamics approximation to our model architecture to capture the motion of the robot arm instead of the static scene background, making it possible to learn versatile skill primitives. In the second part, taking inspiration from the recent progress in latent space planning, we propose a gradient-based optimisation method operating within the latent space of a deep generative model for motion planning. Our approach bypasses the traditional computational challenges encountered by established planning algorithms, and has the capability to specify novel constraints easily and handle multiple constraints simultaneously. Moreover, the training data comes from simple random motor-babbling of kinematically feasible robot states. Our real-world experiments further illustrate that our latent space planning approach can handle both open and closed-loop planning in challenging environments such as heavily cluttered or dynamic scenes. This leads to the first, to our knowledge, closed-loop motion planning algorithm that can incorporate novel custom constraints, and lays the foundation for more complex manipulation tasks

Inverse Kinematic Analysis of Robot Manipulators

An important part of industrial robot manipulators is to achieve desired position and orientation of end effector or tool so as to complete the pre-specified task. To achieve the above stated goal one should have the sound knowledge of inverse kinematic problem. The problem of getting inverse kinematic solution has been on the outline of various researchers and is deliberated as thorough researched and mature problem. There are many fields of applications of robot manipulators to execute the given tasks such as material handling, pick-n-place, planetary and undersea explorations, space manipulation, and hazardous field etc. Moreover, medical field robotics catches applications in rehabilitation and surgery that involve kinematic, dynamic and control operations. Therefore, industrial robot manipulators are required to have proper knowledge of its joint variables as well as understanding of kinematic parameters. The motion of the end effector or manipulator is controlled by their joint actuator and this produces the required motion in each joints. Therefore, the controller should always supply an accurate value of joint variables analogous to the end effector position. Even though industrial robots are in the advanced stage, some of the basic problems in kinematics are still unsolved and constitute an active focus for research. Among these unsolved problems, the direct kinematics problem for parallel mechanism and inverse kinematics for serial chains constitute a decent share of research domain. The forward kinematics of robot manipulator is simpler problem and it has unique or closed form solution. The forward kinematics can be given by the conversion of joint space to Cartesian space of the manipulator. On the other hand inverse kinematics can be determined by the conversion of Cartesian space to joint space. The inverse kinematic of the robot manipulator does not provide the closed form solution. Hence, industrial manipulator can achieve a desired task or end effector position in more than one configuration. Therefore, to achieve exact solution of the joint variables has been the main concern to the researchers. A brief introduction of industrial robot manipulators, evolution and classification is presented. The basic configurations of robot manipulator are demonstrated and their benefits and drawbacks are deliberated along with the applications. The difficulties to solve forward and inverse kinematics of robot manipulator are discussed and solution of inverse kinematic is introduced through conventional methods. In order to accomplish the desired objective of the work and attain the solution of inverse kinematic problem an efficient study of the existing tools and techniques has been done. A review of literature survey and various tools used to solve inverse kinematic problem on different aspects is discussed. The various approaches of inverse kinematic solution is categorized in four sections namely structural analysis of mechanism, conventional approaches, intelligence or soft computing approaches and optimization based approaches. A portion of important and more significant literatures are thoroughly discussed and brief investigation is made on conclusions and gaps with respect to the inverse kinematic solution of industrial robot manipulators. Based on the survey of tools and techniques used for the kinematic analysis the broad objective of the present research work is presented as; to carry out the kinematic analyses of different configurations of industrial robot manipulators. The mathematical modelling of selected robot manipulator using existing tools and techniques has to be made for the comparative study of proposed method. On the other hand, development of new algorithm and their mathematical modelling for the solution of inverse kinematic problem has to be made for the analysis of quality and efficiency of the obtained solutions. Therefore, the study of appropriate tools and techniques used for the solution of inverse kinematic problems and comparison with proposed method is considered. Moreover, recommendation of the appropriate method for the solution of inverse kinematic problem is presented in the work. Apart from the forward kinematic analysis, the inverse kinematic analysis is quite complex, due to its non-linear formulations and having multiple solutions. There is no unique solution for the inverse kinematics thus necessitating application of appropriate predictive models from the soft computing domain. Artificial neural network (ANN) can be gainfully used to yield the desired results. Therefore, in the present work several models of artificial neural network (ANN) are used for the solution of the inverse kinematic problem. This model of ANN does not rely on higher mathematical formulations and are adept to solve NP-hard, non-linear and higher degree of polynomial equations. Although intelligent approaches are not new in this field but some selected models of ANN and their hybridization has been presented for the comparative evaluation of inverse kinematic. The hybridization scheme of ANN and an investigation has been made on accuracies of adopted algorithms. On the other hand, any Optimization algorithms which are capable of solving various multimodal functions can be implemented to solve the inverse kinematic problem. To overcome the problem of conventional tool and intelligent based method the optimization based approach can be implemented. In general, the optimization based approaches are more stable and often converge to the global solution. The major problem of ANN based approaches are its slow convergence and often stuck in local optimum point. Therefore, in present work different optimization based approaches are considered. The formulation of the objective function and associated constrained are discussed thoroughly. The comparison of all adopted algorithms on the basis of number of solutions, mathematical operations and computational time has been presented. The thesis concludes the summary with contributions and scope of the future research work

Learning inverse kinematics with structured prediction

Abstract — Learning inverse kinematics of robots with redundant degrees of freedom (DoF) is a difficult problem in robot learning. The difficulty lies in the non-uniqueness of the inverse kinematics function. Existing methods tackle non-uniqueness by segmenting the configuration space and building a global solution from local experts. The usage of local experts implies the definition of an oracle, which governs the global consistency of the local models; the definition of this oracle is difficult. We propose an algorithm suitable to learn the inverse kinematics function in a single global model despite its multivalued nature. Inverse kinematics is approximated from examples using structured output learning methods. Unlike most of the existing methods, which estimate inverse kinematics on velocity level, we address the learning of the direct function on position level. This problem is a significantly harder. To support the proposed method, we conducted real world experiments on a tracking control task and tested our algorithms on these models. I