End-to-End Learning of Hybrid Inverse Dynamics Models for Precise and Compliant Impedance Control

It is well-known that inverse dynamics models can improve tracking performance in robot control. These models need to precisely capture the robot dynamics, which consist of well-understood components, e.g., rigid body dynamics, and effects that remain challenging to capture, e.g., stick-slip friction and mechanical flexibilities. Such effects exhibit hysteresis and partial observability, rendering them, particularly challenging to model. Hence, hybrid models, which combine a physical prior with data-driven approaches are especially well-suited in this setting. We present a novel hybrid model formulation that enables us to identify fully physically consistent inertial parameters of a rigid body dynamics model which is paired with a recurrent neural network architecture, allowing us to capture unmodeled partially observable effects using the network memory. We compare our approach against state-of-the-art inverse dynamics models on a 7 degree of freedom manipulator. Using data sets obtained through an optimal experiment design approach, we study the accuracy of offline torque prediction and generalization capabilities of joint learning methods. In control experiments on the real system, we evaluate the model as a feed-forward term for impedance control and show the feedback gains can be drastically reduced to achieve a given tracking accuracy

Optimal control and numerical software: an overview

Optimal Control (OC) is the process of determining control and state trajectories for a dynamic system, over a period of time, in order to optimize a given performance index. With the increasing of variables and complexity, OC problems can no longer be solved analytically and, consequently, numerical methods are required. For this purpose, direct and indirect methods are used. Direct methods consist in the discretization of the OC problem, reducing it to a nonlinear constrained optimization problem. Indirect methods are based on the Pontryagin Maximum Principle, which in turn reduces to a boundary value problem. In order to have a more reliable solution, one can solve the same problem through different approaches. Here, as an illustrative example, an epidemiological application related to the rubella disease is solved using several software packages, such as the routine ode45 of Matlab, OC-ODE, DOTcvp toolbox, IPOPT and Snopt, showing the state of the art of numerical software for OC.(undefined

Refraction-corrected ray-based inversion for three-dimensional ultrasound tomography of the breast

Ultrasound Tomography has seen a revival of interest in the past decade, especially for breast imaging, due to improvements in both ultrasound and computing hardware. In particular, three-dimensional ultrasound tomography, a fully tomographic method in which the medium to be imaged is surrounded by ultrasound transducers, has become feasible. In this paper, a comprehensive derivation and study of a robust framework for large-scale bent-ray ultrasound tomography in 3D for a hemispherical detector array is presented. Two ray-tracing approaches are derived and compared. More significantly, the problem of linking the rays between emitters and receivers, which is challenging in 3D due to the high number of degrees of freedom for the trajectory of rays, is analysed both as a minimisation and as a root-finding problem. The ray-linking problem is parameterised for a convex detection surface and three robust, accurate, and efficient ray-linking algorithms are formulated and demonstrated. To stabilise these methods, novel adaptive-smoothing approaches are proposed that control the conditioning of the update matrices to ensure accurate linking. The nonlinear UST problem of estimating the sound speed was recast as a series of linearised subproblems, each solved using the above algorithms and within a steepest descent scheme. The whole imaging algorithm was demonstrated to be robust and accurate on realistic data simulated using a full-wave acoustic model and an anatomical breast phantom, and incorporating the errors due to time-of-flight picking that would be present with measured data. This method can used to provide a low-artefact, quantitatively accurate, 3D sound speed maps. In addition to being useful in their own right, such 3D sound speed maps can be used to initialise full-wave inversion methods, or as an input to photoacoustic tomography reconstructions

Multiresolution strategies for the numerical solution of optimal control problems

Optimal control problems are often characterized by discontinuities or switchings in the control variables. One way of accurately capturing the irregularities in the solution is to use a high resolution (dense) uniform grid. This requires a large amount of computational resources both in terms of CPU time and memory. Hence, in order to accurately capture any irregularities in the solution using a few computational resources, one can refine the mesh locally in the region close to an irregularity instead of refining the mesh uniformly over the whole domain. Therefore, a novel multiresolution scheme for data compression has been designed which is shown to outperform similar data compression schemes. Specifically, we have shown that the proposed approach results in fewer grid points in the grid compared to a common multiresolution data compression scheme. The validity of the proposed mesh refinement algorithm has been verified by solving several challenging initial-boundary value problems for evolution equations in 1D. The examples have demonstrated the stability and robustness of the proposed algorithm. Next, a direct multiresolution-based approach for solving trajectory optimization problems is developed. The original optimal control problem is transcribed into a nonlinear programming (NLP) problem that is solved using standard NLP codes. The novelty of the proposed approach hinges on the automatic calculation of a suitable, nonuniform grid over which the NLP problem is solved, which tends to increase numerical efficiency and robustness. Control and/or state constraints are handled with ease, and without any additional computational complexity. The proposed algorithm is based on a simple and intuitive method to balance several conflicting objectives, such as accuracy of the solution, convergence, and speed of the computations. The benefits of the proposed algorithm over uniform grid implementations are demonstrated with the help of several nontrivial examples. Furthermore, two sequential multiresolution trajectory optimization algorithms for solving problems with moving targets and/or dynamically changing environments have been developed.Ph.D.Committee Chair: Tsiotras, Panagiotis; Committee Member: Calise, Anthony J.; Committee Member: Egerstedt, Magnus; Committee Member: Prasad, J. V. R.; Committee Member: Russell, Ryan P.; Committee Member: Zhou, Hao-Mi

Learning neural ordinary differential equations for optimal control

Ce mémoire rassemble des éléments d'optimisation, d'apprentissage profond et de contrôle optimal afin de répondre aux problématiques d'apprentissage et de planification dans le contexte des systèmes dynamiques en temps continu. Deux approches générales sont explorées. D'abord, une approche basée sur la méthode du maximum de vraisemblance est présentée. Ici, les trajectoires ``d'entrainement'' sont échantillonnées depuis la dynamique réelle, et à partir de celles-ci un modèle de prédiction des états observés est appris. Une fois que l'apprentissage est terminé, le modèle est utilisé pour la planification, en utilisant la dynamique de l'environnement et une fonction de coût pour construire un programme non linéaire, qui est par la suite résolu pour trouver une séquence de contrôle optimal. Ensuite, une approche de bout en bout est proposée, dans laquelle la tâche d'apprentissage de modèle dynamique et celle de planification se déroulent simultanément. Ceci est illustré dans le cadre d'un problème d'apprentissage par imitation, où le modèle est mis à jour en rétropropageant le signal de perte à travers l'algorithme de planification. Grâce au fait que l'entrainement est effectué de bout en bout, cette technique pourrait constituer un sous-module de réseau de neurones de plus grande taille, et pourrait être utilisée pour fournir un biais inductif en faveur des comportements optimaux dans le contexte de systèmes dynamiques en temps continu. Ces méthodes sont toutes les deux conçues pour fonctionner avec des modèles d'équations différentielles ordinaires paramétriques et neuronaux. Également, inspiré par des applications réelles pertinentes, un large recueil de systèmes dynamiques et d'optimiseurs de trajectoire, nommé Myriad, est implémenté; les algorithmes sont testés et comparés sur une variété de domaines de la suite Myriad.This thesis brings together elements of optimization, deep learning and optimal control to study the challenge of learning and planning in continuous-time dynamical systems. Two general approaches are explored. First, a maximum likelihood approach is presented, in which training trajectories are sampled from the true dynamics, and a model is learned to accurately predict the state observations. After training is completed, the learned model is then used for planning, by using the dynamics and cost function to construct a nonlinear program, which can be solved to find a sequence of optimal controls. Second, a fully end-to-end approach is proposed, in which the tasks of model learning and planning are performed simultaneously. This is demonstrated in an imitation learning setting, in which the model is updated by backpropagating the loss signal through the planning algorithm itself. Importantly, because it can be trained in an end-to-end fashion, this technique can be included as a sub-module of a larger neural network, and used to provide an inductive bias towards behaving optimally in a continuous-time dynamical system. Both the maximum likelihood and end-to-end methods are designed to work with parametric and neural ordinary differential equation models. Inspired by relevant real-world applications, a large repository of dynamical systems and trajectory optimizers, named Myriad, is also implemented. The algorithms are tested and compared on a variety of domains within the Myriad suite