Newton method for stochastic control problems

We develop a new iterative method based on Pontryagin principle to solve stochastic control problems. This method is nothing else than the Newton method extended to the framework of stochastic controls, where the state dynamics is given by an ODE with stochastic coefficients. Each iteration of the method is made of two ingredients: computing the Newton direction, and finding an adapted step length. The Newton direction is obtained by solving an affine-linear Forward-Backward Stochastic Differential Equation (FBSDE) with random coefficients. This is done in the setting of a general filtration. We prove that solving such an FBSDE reduces to solving a Riccati Backward Stochastic Differential Equation (BSDE) and an affine-linear BSDE, as expected in the framework of linear FBSDEs or Linear-Quadratic stochastic control problems. We then establish convergence results for this Newton method. In particular, sufficient regularity of the second-order derivative of the cost functional is required to obtain (local) quadratic convergence. A restriction to the space of essentially bounded stochastic processes is needed to obtain such regularity. To choose an appropriate step length while fitting our choice of space of processes, an adapted backtracking line-search method is developed. We then prove global convergence of the Newton method with the proposed line-search procedure, which occurs at a quadratic rate after finitely many iterations. An implementation with regression techniques to solve BSDEs arising in the computation of the Newton step is developed. We apply it to the control problem of a large number of batteries providing ancillary services to an electricity network

AN INTELLIGENT NAVIGATION SYSTEM FOR AN AUTONOMOUS UNDERWATER VEHICLE

The work in this thesis concerns with the development of a novel multisensor data fusion (MSDF) technique, which combines synergistically Kalman filtering, fuzzy logic and genetic algorithm approaches, aimed to enhance the accuracy of an autonomous underwater vehicle (AUV) navigation system, formed by an integration of global positioning system and inertial navigation system (GPS/INS). The Kalman filter has been a popular method for integrating the data produced by the GPS and INS to provide optimal estimates of AUVs position and attitude. In this thesis, a sequential use of a linear Kalman filter and extended Kalman filter is proposed. The former is used to fuse the data from a variety of INS sensors whose output is used as an input to the later where integration with GPS data takes place. The use of an adaptation scheme based on fuzzy logic approaches to cope with the divergence problem caused by the insufficiently known a priori filter statistics is also explored. The choice of fuzzy membership functions for the adaptation scheme is first carried out using a heuristic approach. Single objective and multiobjective genetic algorithm techniques are then used to optimize the parameters of the membership functions with respect to a certain performance criteria in order to improve the overall accuracy of the integrated navigation system. Results are presented that show that the proposed algorithms can provide a significant improvement in the overall navigation performance of an autonomous underwater vehicle navigation. The proposed technique is known to be the first method used in relation to AUV navigation technology and is thus considered as a major contribution thereof.J&S Marine Ltd., Qinetiq, Subsea 7 and South West Water PL

Imitation Learning of Motion Coordination in Robots:a Dynamical System Approach

The ease with which humans coordinate all their limbs is fascinating. Such a simplicity is the result of a complex process of motor coordination, i.e. the ability to resolve the biomechanical redundancy in an efficient and repeatable manner. Coordination enables a wide variety of everyday human activities from filling in a glass with water to pair figure skating. Therefore, it is highly desirable to endow robots with similar skills. Despite the apparent diversity of coordinated motions, all of them share a crucial similarity: these motions are dictated by underlying constraints. The constraints shape the formation of the coordination patterns between the different degrees of freedom. Coordination constraints may take a spatio-temporal form; for instance, during bimanual object reaching or while catching a ball on the fly. They also may relate to the dynamics of the task; for instance, when one applies a specific force profile to carry a load. In this thesis, we develop a framework for teaching coordination skills to robots. Coordination may take different forms, here, we focus on teaching a robot intra-limb and bimanual coordination, as well as coordination with a human during physical collaborative tasks. We use tools from well-established domains of Bayesian semiparametric learning (Gaussian Mixture Models and Regression, Hidden Markov Models), nonlinear dynamics, and adaptive control. We take a biologically inspired approach to robot control. Specifically, we adopt an imitation learning perspective to skill transfer, that offers a seamless and intuitive way of capturing the constraints contained in natural human movements. As the robot is taught from motion data provided by a human teacher, we exploit evidence from human motor control of the temporal evolution of human motions that may be described by dynamical systems. Throughout this thesis, we demonstrate that the dynamical system view on movement formation facilitates coordination control in robots. We explain how our framework for teaching coordination to a robot is built up, starting from intra-limb coordination and control, moving to bimanual coordination, and finally to physical interaction with a human. The dissertation opens with the discussion of learning discrete task-level coordination patterns, such as spatio-temporal constraints emerging between the two arms in bimanual manipulation tasks. The encoding of bimanual constraints occurs at the task level and proceeds through a discretization of the task as sequences of bimanual constraints. Once the constraints are learned, the robot utilizes them to couple the two dynamical systems that generate kinematic trajectories for the hands. Explicit coupling of the dynamical systems ensures accurate reproduction of the learned constraints, and proves to be crucial for successful accomplishment of the task. In the second part of this thesis, we consider learning one-arm control policies. We present an approach to extracting non-linear autonomous dynamical systems from kinematic data of arbitrary point-to-point motions. The proposed method aims to tackle the fundamental questions of learning robot coordination: (i) how to infer a motion representation that captures a multivariate coordination pattern between degrees of freedom and that generalizes this pattern to unseen contexts; (ii) whether the policy learned directly from demonstrations can provide robustness against spatial and temporal perturbations. Finally, we demonstrate that the developed dynamical system approach to coordination may go beyond kinematic motion learning. We consider physical interactions between a robot and a human in situations where they jointly perform manipulation tasks; in particular, the problem of collaborative carrying and positioning of a load. We extend the approach proposed in the second part of this thesis to incorporate haptic information into the learning process. As a result, the robot adapts its kinematic motion plan according to human intentions expressed through the haptic signals. Even after the robot has learned the task model, the human still remains a complex contact environment. To ensure robustness of the robot behavior in the face of the variability inherent to human movements, we wrap the learned task model in an adaptive impedance controller with automatic gain tuning. The techniques, developed in this thesis, have been applied to enable learning of unimanual and bimanual manipulation tasks on the robotics platforms HOAP-3, KATANA, and i-Cub, as well as to endow a pair of simulated robots with the ability to perform a manipulation task in the physical collaboration

Stabilizer architecture for humanoid robots collaborating with humans

Hoy en día, los avances en las tecnologías de información y comunicación permiten el uso de robots como compañeros en las actividades con los seres humanos. Mientras que la mayoría de las investigaciones existentes se dedica a la interacción entre humanos y robots, el marco de esta investigación está centrado en el uso de robots como agentes de colaboración. En particular, este estudio está dedicado a los robots humanoides que puedan ayudar a la gente en varias tareas en entornos de trabajo. Los robots humanoides son sin duda los m as adecuados para este tipo de situaciones: pueden usar las mismas herramientas que los seres humanos y son lo m as probablemente aceptados por ellos. Después de explicar las ventajas de las tareas de colaboración entre los humanos y los robots y las diferencias con respecto a los sistemas de interacción y de teleoperación, este trabajo se centra en el nivel de las tecnologías que es necesario para lograr ese objetivo. El problema más complicado en el control de humanoides es el balance de la estructura. Este estudio se centra en técnicas novedosas para la estimación de la actitud del robot, que se utilizarán para el control. El control del robot se basa en un modelo muy conocido y simplificado: el péndulo invertido. Este modelo permite tener un control en tiempo real sobre la estructura, mientras que esté sometida a fuerzas externas / disturbios. Trayectorias suaves para el control de humanoides se han propuesto y probado en plataformas reales: éstos permiten reducir los impactos del robot con su entorno. Finalmente, el estudio extiende estos resultados a una contribución para la arquitectura de colaboración humano-humanoide. Dos tipos de colaboraciones humano humanoide se analizan: la colaboración física, donde robots y humanos comparten el mismo espacio y tienen un contacto físico (o por medio de un objeto), y una colaboración a distancia, en la que el ser humano está relativamente lejos del robot y los dos agentes colaboran por medio de una interfaz. El paradigma básico de esta colaboración robótica es: lo que es difícil (o peligroso) para el ser humano se hace por medio del robot y lo que es difícil para el robot lo puede mejor hacer el humano. Es importante destacar que el contexto de los experimentos no se basa en una unica plataforma humanoide; por el contrario, tres plataformas han sido objeto de los experimentos: se han empleado los robots HOAP-3, HRP-2 y TEO. ----------------------------------------------------------------------------------------------------------------------------------------------------------Nowadays, the advances in information and communication technologies permit the use of robots as companions in activities with humans. While most of the existing research is dedicated to the interaction between humans and robots, the framework of this research is the use of robots as collaborative agents. In particular, this study is dedicated to humanoid robots which should assist people in several tasks in working environments. Humanoid robots are certainly the most adequate for such situations: they can use the same tools as humans and are most likely accepted by them. After explaining the advantages of collaborative tasks among humans and robots and the differences with respect to interaction and teleoperation systems, this work focuses on the level of technologies which is necessary in order to achieve such a goal. The most complicated problem in humanoid control is the structure balance. This study focuses in novel techniques in the attitude estimation of the robot, to be used for the control. The control of the robot is based on a very well-known and simplified model: the double inverted pendulum. This model permits having a real-time control on the structure while submitted to external forces/disturbances. The control actions are strongly dependent on the three stability regions, which are determined by the position of the ZMP in the support polygon. Smooth trajectories for the humanoid control have been proposed and tested on real platforms: these permit reducing the impacts of the robot with its environment. Finally, the study extends these results to a contribution for human-humanoid collaboration architecture. Two types of human-humanoid collaborations are analyzed: a physical collaboration, where robot and human share the same space and have a physical contact (or by means of an object), and a remote collaboration, in which the human is relatively far away from the robot and the two agents collaborate using an interface. The basic paradigm for this robotic collaboration is: what is difficult (or dangerous) for the human is done by the robot and what is difficult for the robot is better done by the human. Importantly, the testing context is not based on a single humanoid platform; on the contrary, three platforms have been object of the experiments: the Hoap-3, HRP-2 and HRP2 robot have been employed

Interferometric study of density fluctuations in a tokamak plasma

Density fluctuations in the LT-4 tokamak plasma are investigated using a Phase Scintillation Interferometer operating at 10.6/Ltm which is sensitive to density fluctuations of δnₑ/nₑ> 10⁻¹⁴. The plasma is imaged across a linear detector array which can be rotated to record projections in any direction, from toroidal to poloidal. The theory of forward scattering from plasmas is developed from the Rytov approximation and aspects of the Fourier diffraction projection theorem relevant to plasma scattering. The result is a clear conceptual picture of diffraction from arbitrary extended refractive media, from which important analytical tools are developed. The Phase Scintillation Interferometer is used to image density perturbations produced by large scale magnetohydro dynamic (MHD) modes in the plasma associated with Mimov oscillations. Structural characteristics are determined, and a comparison between experimental and computed projections of the Dubois model is made which shows that the density fluctuations are consistent with a model of rotating magnetic islands. Island widths and local magnetic field fluctuations are determined and are found to compare well with measured poloidal magnetic field fluctuations. The interferometer is used in conjunction with other diagnostics to investigate minor and major disruptions in LT-4. The time frequency distribution is introduced as an important analytical tool in the characterization of the various regimes of MHD activity. Frequency and amplitude variations of an m = 3 mode during current rise appear correlated with variations in toroidal loop voltage. The mode is also found to persist throughout the whole discharge and to play a part in mode locking which precedes major disruptions. Mode frequencies are found to vary in a regular way with the safety factor q(a). Precursor oscillations before minor and major disruptions are identified. A strong m — 1 type of internal relaxation is found to follow rapid growth and locking of an m = 2 mode during minor disruptions. The interferometer is also applied to the measurement of fine scale density fluctuations in the LT-4 tokamak during periods of low level MHD activity. Line integral measurements indicate an edge fluctuation level of about 10% and broad band spectra typical of strong turbulence. Anisotropy in the spectrum of fluctuations perpendicular to the magnetic field is observed. This observation runs counter to reported measurements of isotropic fluctuations made on other tokamaks using small angle scattering techniques. Very long correlation lengths along the field lines are observed, which are consistent with nearly all models of turbulence in tokamak plasmas. The images are numerically filtered so as to isolate and display counter-propagating structures in the turbulent flow

Application of nonsmooth optimisation to data analysis

The research presented in this thesis is two-fold: on the one hand, major data mining problems are reformulated as mathematical programming problems. These problems should be carefully designed, since from their formulation depends the efficiency, perhaps the existence, of the solvers. On the other hand, optimisation methods are adapted to solve these problems, most of which are nonsmooth and nonconvex. This part is delicate, as the solution is often required to be good and obtained fast. Numerical experiments on real-world datasets are presented and analysed.Doctor of Philosoph