A Zero-Shot Adaptive Quadcopter Controller

This paper proposes a universal adaptive controller for quadcopters, which can be deployed zero-shot to quadcopters of very different mass, arm lengths and motor constants, and also shows rapid adaptation to unknown disturbances during runtime. The core algorithmic idea is to learn a single policy that can adapt online at test time not only to the disturbances applied to the drone, but also to the robot dynamics and hardware in the same framework. We achieve this by training a neural network to estimate a latent representation of the robot and environment parameters, which is used to condition the behaviour of the controller, also represented as a neural network. We train both networks exclusively in simulation with the goal of flying the quadcopters to goal positions and avoiding crashes to the ground. We directly deploy the same controller trained in the simulation without any modifications on two quadcopters with differences in mass, inertia, and maximum motor speed of up to 4 times. In addition, we show rapid adaptation to sudden and large disturbances (up to 35.7%) in the mass and inertia of the quadcopters. We perform an extensive evaluation in both simulation and the physical world, where we outperform a state-of-the-art learning-based adaptive controller and a traditional PID controller specifically tuned to each platform individually. Video results can be found at https://dz298.github.io/universal-drone-controller/.Comment: Video results can be found on the project webpage https://dz298.github.io/universal-drone-controller

Sub-Optimality of a Dyadic Adaptive Control Architecture

The dyadic adaptive control architecture evolved as a solution to the problem of designing control laws for nonlinear systems with unmatched nonlinearities, disturbances and uncertainties. A salient feature of this framework is its ability to work with infinite as well as finite dimensional systems, and with a wide range of control and adaptive laws. In this paper, we consider the case where a control law based on the linear quadratic regulator theory is employed for designing the control law. We benchmark the closed-loop system against standard linear quadratic control laws as well as those based on the state-dependent Riccati equation. We pose the problem of designing a part of the control law as a Nehari problem. We obtain analytical expressions for the bounds on the sub-optimality of the control law

Data-Driven Robust Reinforcement Learning Control of Uncertain Nonlinear Systems: Towards a Fully-Automated, Insulin-Based Artificial Pancreas

In this paper, a novel robust tracking control scheme for a general class of discrete-time nonlinear systems affected by unknown bounded uncertainty is presented. By solving a parameterized optimal tracking control problem subject to the unknown nominal system and a suitable cost function, the resulting optimal tracking control policy can ensure closed-loop stability by achieving a sufficiently small tracking error for the original uncertain nonlinear system. The computation of the optimal tracking controller is accomplished through the derivation of a novel Q-function-based $\lambda$-Policy Iteration algorithm. The proposed algorithm not only enjoys rigorous theoretical guarantees, but also avoids technical weaknesses of conventional reinforcement learning methods. By employing a data-driven, critic-only least squares implementation, the performance of the proposed algorithm is evaluated to the problem of fully-automated, insulin-based, closed-loop glucose control for patients diagnosed with Type 1 and Type 2 Diabetes Mellitus. The U.S. FDA-accepted DMMS.R simulator from the Epsilon Group is used to conduct a comprehensive in silico clinical campaign on a rich set of virtual subjects under completely unannounced meal and exercise settings. Simulation results underline the superior glycaemic behavior achieved by the derived approach, as well as its overall maturity for the design of highly-effective, closed-loop drug delivery systems for personalized medicine

Integral sliding modes generation via DRL-assisted Lyapunov-based control for robot manipulators

This paper proposes an enhanced version of the integral sliding mode (ISM) control, where a deep neural network (DNN) is first trained as a deep reinforcement learning (DRL) agent. Then, such a DNN is fine-tuned relying on a Lyapunov-based weight adaptation law, with the aim of compensating the lack of knowledge of the full dynamics in the case of robot manipulators. Specifically, a DRL agent is trained off-line on a reward depending on the sliding variable to estimate the unknown drift term of the robot dynamics. Such an estimate is then exploited to initialize and perform the fine tuning of the online adaptation mechanism based on the DNN. The proposal is theoretically analysed and assessed in simulation relying on the planar configuration of a Franka Emika Panda robot manipulator

Decentralised State Feedback Tracking Control for Large-Scale Interconnected Systems Using Sliding Mode Techniques

A class of large-scale interconnected systems with matched and unmatched uncertainties is studied in this thesis, with the objective of proposing a controller based on diffeomorphisms and some techniques to deal with the tracking problem of the system. The main research developed in this thesis includes: 1. Large-scale interconnected system is a complex system consisting of several semi-independent subsystems, which are typically located in distinct geographic or logical locations. In this situation, decentralised control which only collects the local information is the practical method to deal with large-scale interconnected systems. The decentralised methodology is utilised throughout this thesis, guaranteeing that systems exhibit essential robustness against uncertainty. 2. Sliding mode technique is involved in the process of controller design. By introducing a nonsingular local diffeomorphism, the large-scale system can be transformed into a system with a specific regular form, where the matched uncertainty is completely absent from the subspace spanned by the sliding mode dynamics. The sliding mode based controller is proposed in this thesis to successfully achieve high robustness of the closed-loop interconnected systems with some particular uncertainties. 3. The considered large-scale interconnected systems can always track the smooth desired signals in a finite time. Each subsystem can track its own ideal signal or all subsystems can track the same ideal signal. Both situations are discussed in this thesis and the results are mathematically proven by introducing the Lyapunov theory, even when operating under the presence of disturbances. At the end of each chapter, some simulation examples, like a coupled inverted pendulums system, a river pollution system and a high-speed train system, are presented to verify the correctness of the proposed theory. At the conclusion of this thesis, a brief summary of the research findings has been provided, along with a mention of potential future research directions in tracking control of large-scale systems, like more general boundedness of interconnections, possibilities of distributed control, collaboration with intelligent control and so on. Some mathematical theories involved and simulation code are included in the appendix section

Bio-inspired robotic control in underactuation: principles for energy efficacy, dynamic compliance interactions and adaptability.

Biological systems achieve energy efficient and adaptive behaviours through extensive autologous and exogenous compliant interactions. Active dynamic compliances are created and enhanced from musculoskeletal system (joint-space) to external environment (task-space) amongst the underactuated motions. Underactuated systems with viscoelastic property are similar to these biological systems, in that their self-organisation and overall tasks must be achieved by coordinating the subsystems and dynamically interacting with the environment. One important question to raise is: How can we design control systems to achieve efficient locomotion, while adapt to dynamic conditions as the living systems do? In this thesis, a trajectory planning algorithm is developed for underactuated microrobotic systems with bio-inspired self-propulsion and viscoelastic property to achieve synchronized motion in an energy efficient, adaptive and analysable manner. The geometry of the state space of the systems is explicitly utilized, such that a synchronization of the generalized coordinates is achieved in terms of geometric relations along the desired motion trajectory. As a result, the internal dynamics complexity is sufficiently reduced, the dynamic couplings are explicitly characterised, and then the underactuated dynamics are projected onto a hyper-manifold. Following such a reduction and characterization, we arrive at mappings of system compliance and integrable second-order dynamics with the passive degrees of freedom. As such, the issue of trajectory planning is converted into convenient nonlinear geometric analysis and optimal trajectory parameterization. Solutions of the reduced dynamics and the geometric relations can be obtained through an optimal motion trajectory generator. Theoretical background of the proposed approach is presented with rigorous analysis and developed in detail for a particular example. Experimental studies are conducted to verify the effectiveness of the proposed method. Towards compliance interactions with the environment, accurate modelling or prediction of nonlinear friction forces is a nontrivial whilst challenging task. Frictional instabilities are typically required to be eliminated or compensated through efficiently designed controllers. In this work, a prediction and analysis framework is designed for the self-propelled vibro-driven system, whose locomotion greatly relies on the dynamic interactions with the nonlinear frictions. This thesis proposes a combined physics-based and analytical-based approach, in a manner that non-reversible characteristic for static friction, presliding as well as pure sliding regimes are revealed, and the frictional limit boundaries are identified. Nonlinear dynamic analysis and simulation results demonstrate good captions of experimentally observed frictional characteristics, quenching of friction-induced vibrations and satisfaction of energy requirements. The thesis also performs elaborative studies on trajectory tracking. Control schemes are designed and extended for a class of underactuated systems with concrete considerations on uncertainties and disturbances. They include a collocated partial feedback control scheme, and an adaptive variable structure control scheme with an elaborately designed auxiliary control variable. Generically, adaptive control schemes using neural networks are designed to ensure trajectory tracking. Theoretical background of these methods is presented with rigorous analysis and developed in detail for particular examples. The schemes promote the utilization of linear filters in the control input to improve the system robustness. Asymptotic stability and convergence of time-varying reference trajectories for the system dynamics are shown by means of Lyapunov synthesis