Onboard Robust Nonlinear Control for Multiple Multirotor UAVs

In this thesis, we focus on developing and validating onboard robust nonlinear control approaches for multiple multirotor unmanned aerial vehicles (UAVs), for the promise of achieving nontrivial tasks, such as path following with aggressive maneuvers, navigation in complex environments with obstacles, and formation in group. To fulfill these challenging missions, the first concern comes with the stability of flight control for the aggressive UAV maneuvers with large tilted angles. In addition, robustness of control is highly desired in order to lead the multirotor UAVs to safe and accurate performance under disturbances. Furthermore, efficiency of control algorithm is a crucial element for the onboard implementation with limited computational capability. Finally, the potential to simultaneously control a group of UAVs in a stable fashion is required. All of these concerns motivate our work in this thesis in the following aspects. We first propose the problem of robust control for the nontrivial maneuvers of a multirotor UAV under disturbances. A complete framework is developed to enable the UAV to achieve the challenging tasks, which consists of a nonlinear attitude controller based on the solution of global output regulation problems for the rigid body rotations SO(3), a backstepping-like position controller, a six-dimensional (6D) wrench observer to estimate the unknown force and torque disturbances, and an online trajectory planner based on a model predictive control (MPC) method. We prove the strong convergence properties of the proposed method both in theory and via intensive real-robot experiments of aggressive waypoint navigation and large-tilted path following tasks in the presence of external disturbances, e.g. wind gusts. Secondly, we propose the problem of autonomous navigation of a multirotor UAV in complex scenarios. We present an effective and robust control approach, namely a fast MPC method with the inclusion of nonlinear obstacle avoiding constraints, and implement it onboard the UAV at 50Hz. The developed approach enables the navigation for a multirotor UAV in 3D environments with multiple obstacles, by autonomously deciding to fly over or around the randomly located obstacles. The third problem that is addressed in our work is formation control for a group of multirotor UAVs. We solve this problem by proposing a distributed formation control algorithm for multiple UAVs based on the solution of retraction balancing problem. The algorithm brings the whole group of UAVs simultaneously to a prescribed submanifold that determines the formation shape in an asymptotically stable fashion in 2D and 3D environments. We validate our proposed algorithm via a series of hardware-in-the-loop simulations and real-robot experiments in various formation cases of arbitrary time-varying (e.g. expanding, shrinking or moving) shapes. In the actual experiments, up to 4 multirotors have been implemented to form arbitrary triangular, rectangular and circular shapes drawn by the operator via a human-robot-interaction device. We have also carried out virtual tests using up to 6 onboard computers to achieve a spherical formation and a formation moving through obstacles.In dieser Arbeit konzentrieren wir uns auf die Entwicklung und Validierung von robusten nichtlinearen On-Bord Steuerungsansatzen für mehrere unbemannte Multirotor-Luftfahrzeuge (UAVs), mit dem Ziel, nicht triviale Aufgaben zu erledigen wie z.B. Wegfolge mit aggressiven Manovern, Navigation in komplexen Umgebungen mit Hindernissen und Formationsflug in einer Gruppe. Um diese anspruchsvollen Missionen zu erfullen liegt unser Hauptaugenmerk bei der Stabilität der Flugsteuerung für aggressive UAV Manöver mit steilen Lagewinkeln. Des weiteren ist Kontroll-robustheit sehr wünschenswert, um die Multirotor-UAVs unter Beeinflussung sicher und genau zu steuern. Daruber hinaus ist die Effizienz des Kontrollalgorithmus ein wichtiges Element für die Onboard-Implementierung mit eingeschrankter Rechenfähigkeit. Abschliessend ist das Potenzial, gleichzeitig eine Gruppe von UAVs in stabiler Weise zu kontrollieren, erforderlich. All dies motiviert uns zur Arbeit an den folgenden Aspekten: Zuerst behandeln wir das Problem der robusten Steuerung nichttrivialer Manöver eines Multirotor UAV unter Störeinfluss. Ein komplettes Framework wird entwickelt, welches dem UAV ermöglicht diese anspruchsvollen Aufgaben zu bewältigen. Es beinhaltet einem nichtlinearen Lageregler, basierend auf der Lösung von globalen Ausgangsrege lungsproblemen für Starrkörperrotationen SO(3), einem backstepping basierten Positionsregler, einen sechsdimensionalen (6D) wrench observer um die unbekannten Kraftund Drehmomenteinflusse zu schätzen, sowie einem Online-Trajektorienplaner basierend auf Model Predictive Control (MPC). Wir weisen die starken Konvergenzcharakteristiken der vorgeschlagenen Methode nach, sowohl in der Theorie als auchmittels intensiver Real-roboter-Experimente, mit aggressiver Wegpunktnavigation und Wegfindungsaufgaben in extremer Fluglage in Gegenwart externer Einflüsse, z.B. Windböen. Als nächstes bearbeiten wir das Problem der autonomen Navigation eines Multirotor UAV in komplexen Szenarien. Wir stellen einen effektiven und robusten Steuerungsansatz dar, nämlich eine schnelle MPC-Methode mit der Einbeziehung von nichtlinearer Einschränkungen zur Hindernisvermeidung, und implmenetieren diese an Bord des UAV mit 50Hz. Der entwickelte Ansatz ermöglicht die Navigation eines Multirotor UAVs in 3D-Umgebungen mit mehreren Hindernissen, wobei autonom entschieden wir, über oder um die zufällig gelegenen Hindernisse zu fliegen. Das dritte Problem, das in unserer Arbeit angesprochen wird, ist die Bildungssteuerung für eine Gruppe von Multirotor UAVs. Wir lösen dieses Problem, indem wir einen verteilten Formationskontrollalgorithmus für mehrere UAVs auf der Grundlage der Lösung des Retraction Balancing Problems vorschlagen. Der Algorithmus bringt die ganze Gruppe von UAVs gleichzeitig auf eine vorgeschriebene Untermanigfaltigkeit, welche die Formation in asymtotisch stabiler Weise in 2D- und 3D-Umgebungen bestimmt. Wir validieren unseren vorgeschlagenen Algorithmus uber eine Reihe von Hardware-in-the- ¨ Loop-Simulationen und Real-Roboter-Experimente mit verschiedenen Formationsvarianten in beliebigen zeitveränderlichen (z. B. expandierenden, schrumpfenden oder bewegten) Formen. In den eigentlichen Experimenten wurden bis zu 4 Multirotoren eingesetzt, um beliebige dreieckige, rechteckige und kreisförmige Formen zu bilden, die vom Bediener über eine Mensch-Roboter-Interaktionsvorrichtung vorgezeichnet wurden. Wir haben auch virtuelle Tests mit bis zu 6 Onboard-Computern durchgeführt, um eine sphärische Formation und eine Formation zu erreichen, die sich durch Hindernisse. bewegt

An adaptive hierarchical control for aerial manipulators

This paper addresses the trajectory tracking control problem for a quadrotor aerial vehicle, equipped with a robotic manipulator (aerial manipulator). The controller is organized in two layers: in the top layer, an inverse kinematics algorithm computes the motion references for the actuated variables; in the bottom layer, a motion control algorithm is in charge of tracking the motion references computed by the upper layer. To the purpose, a model-based control scheme is adopted, where modelling uncertainties are compensated through an adaptive term. The stability of the proposed scheme is proven by resorting to Lyapunov arguments. Finally, a simulation case study is proposed to prove the effectiveness of the approach

Guidance, navigation and control of multirotors

Aplicat embargament des de la data de defensa fins el dia 31 de desembre de 2021This thesis presents contributions to the Guidance, Navigation and Control (GNC) systems for multirotor vehicles by applying and developing diverse control techniques and machine learning theory with innovative results. The aim of the thesis is to obtain a GNC system able to make the vehicle follow predefined paths while avoiding obstacles in the vehicle's route. The system must be adaptable to different paths, situations and missions, reducing the tuning effort and parametrisation of the proposed approaches. The multirotor platform, formed by the Asctec Hummingbird quadrotor vehicle, is studied and described in detail. A complete mathematical model is obtained and a freely available and open simulation platform is built. Furthermore, an autopilot controller is designed and implemented in the real platform. The control part is focused on the path following problem. That is, following a predefined path in space without any time constraint. Diverse control-oriented and geometrical algorithms are studied, implemented and compared. Then, the geometrical algorithms are improved by obtaining adaptive approaches that do not need any parameter tuning. The adaptive geometrical approaches are developed by means of Neural Networks. To end up, a deep reinforcement learning approach is developed to solve the path following problem. This approach implements the Deep Deterministic Policy Gradient algorithm. The resulting approach is trained in a realistic multirotor simulator and tested in real experiments with success. The proposed approach is able to accurately follow a path while adapting the vehicle's velocity depending on the path's shape. In the navigation part, an obstacle detection system based on the use of a LIDAR sensor is implemented. A model of the sensor is derived and included in the simulator. Moreover, an approach for treating the sensor data to eliminate the possible ground detections is developed. The guidance part is focused on the reactive path planning problem. That is, a path planning algorithm that is able to re-plan the trajectory online if an unexpected event, such as detecting an obstacle in the vehicle's route, occurs. A deep reinforcement learning approach for the reactive obstacle avoidance problem is developed. This approach implements the Deep Deterministic Policy Gradient algorithm. The developed deep reinforcement learning agent is trained and tested in the realistic simulation platform. This agent is combined with the path following agent and the rest of the elements developed in the thesis obtaining a GNC system that is able to follow different types of paths while avoiding obstacle in the vehicle's route.Aquesta tesi doctoral presenta diverses contribucions relaciones amb els sistemes de Guiat, Navegació i Control (GNC) per a vehicles multirrotor, aplicant i desenvolupant diverses tècniques de control i de machine learning amb resultats innovadors. L'objectiu principal de la tesi és obtenir un sistema de GNC capaç de dirigir el vehicle perquè segueixi una trajectòria predefinida mentre evita els obstacles que puguin aparèixer en el recorregut del vehicle. El sistema ha de ser adaptable a diferents trajectòries, situacions i missions, reduint l'esforç realitzat en l'ajust i la parametrització dels mètodes proposats. La plataforma experimental, formada pel cuadricòpter Asctec Hummingbird, s'estudia i es descriu en detall. S'obté un model matemàtic complet de la plataforma i es desenvolupa una eina de simulació, la qual és de codi lliure. A més, es dissenya un controlador autopilot i s'implementa en la plataforma real. La part de control està enfocada al problema de path following. En aquest problema, el vehicle ha de seguir una trajectòria predefinida en l'espai sense cap tipus de restricció temporal. S'estudien, s'implementen i es comparen diversos algoritmes de control i geomètrics de path following. Després, es milloren els algoritmes geomètrics usant xarxes neuronals per convertirlos en algoritmes adaptatius. Per finalitzar, es desenvolupa un mètode de path following basat en tècniques d'aprenentatge per reforç profund (deep Reinforcement learning). Aquest mètode implementa l'algoritme Deep Deterministic Policy Gradient. L'agent intel. ligent resultant és entrenat en un simulador realista de multirotors i validat en la plataforma experimental real amb èxit. Els resultats mostren que l'agent és capaç de seguir de forma precisa la trajectòria de referència adaptant la velocitat del vehicle segons la curvatura del recorregut. A la part de navegació, s'implementa un sistema de detecció d'obstacles basat en l'ús d'un sensor LIDAR. Es deriva un model del sensor i aquest s'inclou en el simulador. A més, es desenvolupa un mètode per tractar les mesures del sensor per eliminar les possibles deteccions del terra. Pel que fa a la part de guiatge, aquesta està focalitzada en el problema de reactive path planning. És a dir, un algoritme de planificació de trajectòria que és capaç de re-planejar el recorregut del vehicle a l'instant si algun esdeveniment inesperat ocorre, com ho és la detecció d'un obstacle en el recorregut del vehicle. Es desenvolupa un mètode basat en aprenentatge per reforç profund per l'evasió d'obstacles. Aquest mètode implementa l'algoritme Deep Deterministic Policy Gradient. L'agent d'aprenentatge per reforç s'entrena i valida en un simulador de multirotors realista. Aquest agent es combina amb l'agent de path following i la resta d'elements desenvolupats en la tesi per obtenir un sistema GNC capaç de seguir diferents tipus de trajectòries, evadint els obstacles que estiguin en el recorregut del vehicle.Esta tesis doctoral presenta varias contribuciones relacionas con los sistemas de Guiado, Navegación y Control (GNC) para vehículos multirotor, aplicando y desarrollando diversas técnicas de control y de machine learning con resultados innovadores. El objetivo principal de la tesis es obtener un sistema de GNC capaz de dirigir el vehículo para que siga una trayectoria predefinida mientras evita los obstáculos que puedan aparecer en el recorrido del vehículo. El sistema debe ser adaptable a diferentes trayectorias, situaciones y misiones, reduciendo el esfuerzo realizado en el ajuste y la parametrización de los métodos propuestos. La plataforma experimental, formada por el cuadricoptero Asctec Hummingbird, se estudia y describe en detalle. Se obtiene un modelo matemático completo de la plataforma y se desarrolla una herramienta de simulación, la cual es de código libre. Además, se diseña un controlador autopilot, el cual es implementado en la plataforma real. La parte de control está enfocada en el problema de path following. En este problema, el vehículo debe seguir una trayectoria predefinida en el espacio tridimensional sin ninguna restricción temporal Se estudian, implementan y comparan varios algoritmos de control y geométricos de path following. Luego, se mejoran los algoritmos geométricos usando redes neuronales para convertirlos en algoritmos adaptativos. Para finalizar, se desarrolla un método de path following basado en técnicas de aprendizaje por refuerzo profundo (deep reinforcement learning). Este método implementa el algoritmo Deep Deterministic Policy Gradient. El agente inteligente resultante es entrenado en un simulador realista de multirotores y validado en la plataforma experimental real con éxito. Los resultados muestran que el agente es capaz de seguir de forma precisa la trayectoria de referencia adaptando la velocidad del vehículo según la curvatura del recorrido. En la parte de navegación se implementa un sistema de detección de obstáculos basado en el uso de un sensor LIDAR. Se deriva un modelo del sensor y este se incluye en el simulador. Además, se desarrolla un método para tratar las medidas del sensor para eliminar las posibles detecciones del suelo. En cuanto a la parte de guiado, está focalizada en el problema de reactive path planning. Es decir, un algoritmo de planificación de trayectoria que es capaz de re-planear el recorrido del vehículo al instante si ocurre algún evento inesperado, como lo es la detección de un obstáculo en el recorrido del vehículo. Se desarrolla un método basado en aprendizaje por refuerzo profundo para la evasión de obstáculos. Este implementa el algoritmo Deep Deterministic Policy Gradient. El agente de aprendizaje por refuerzo se entrena y valida en un simulador de multirotors realista. Este agente se combina con el agente de path following y el resto de elementos desarrollados en la tesis para obtener un sistema GNC capaz de seguir diferentes tipos de trayectorias evadiendo los obstáculos que estén en el recorrido del vehículo.Postprint (published version

Adaptive and learning-based formation control of swarm robots

Autonomous aerial and wheeled mobile robots play a major role in tasks such as search and rescue, transportation, monitoring, and inspection. However, these operations are faced with a few open challenges including robust autonomy, and adaptive coordination based on the environment and operating conditions, particularly in swarm robots with limited communication and perception capabilities. Furthermore, the computational complexity increases exponentially with the number of robots in the swarm. This thesis examines two different aspects of the formation control problem. On the one hand, we investigate how formation could be performed by swarm robots with limited communication and perception (e.g., Crazyflie nano quadrotor). On the other hand, we explore human-swarm interaction (HSI) and different shared-control mechanisms between human and swarm robots (e.g., BristleBot) for artistic creation. In particular, we combine bio-inspired (i.e., flocking, foraging) techniques with learning-based control strategies (using artificial neural networks) for adaptive control of multi- robots. We first review how learning-based control and networked dynamical systems can be used to assign distributed and decentralized policies to individual robots such that the desired formation emerges from their collective behavior. We proceed by presenting a novel flocking control for UAV swarm using deep reinforcement learning. We formulate the flocking formation problem as a partially observable Markov decision process (POMDP), and consider a leader-follower configuration, where consensus among all UAVs is used to train a shared control policy, and each UAV performs actions based on the local information it collects. In addition, to avoid collision among UAVs and guarantee flocking and navigation, a reward function is added with the global flocking maintenance, mutual reward, and a collision penalty. We adapt deep deterministic policy gradient (DDPG) with centralized training and decentralized execution to obtain the flocking control policy using actor-critic networks and a global state space matrix. In the context of swarm robotics in arts, we investigate how the formation paradigm can serve as an interaction modality for artists to aesthetically utilize swarms. In particular, we explore particle swarm optimization (PSO) and random walk to control the communication between a team of robots with swarming behavior for musical creation

Survey on Motion Planning for Multirotor Aerial Vehicles in Plan-based Control Paradigm

In general, optimal motion planning can be performed both locally and globally. In such a planning, the choice in favour of either local or global planning technique mainly depends on whether the environmental conditions are dynamic or static. Hence, the most adequate choice is to use local planning or local planning alongside global planning. When designing optimal motion planning both local and global, the key metrics to bear in mind are execution time, asymptotic optimality, and quick reaction to dynamic obstacles. Such planning approaches can address the aforesaid target metrics more efficiently compared to other approaches such as path planning followed by smoothing. Thus, the foremost objective of this study is to analyse related literature in order to understand how the motion planning, especially trajectory planning, problem is formulated, when being applied for generating optimal trajectories in real-time for Multirotor Aerial Vehicles, impacts the listed metrics. As a result of the research, the trajectory planning problem was broken down into a set of subproblems, and the lists of methods for addressing each of the problems were identified and described in detail. Subsequently, the most prominent results from 2010 to 2022 were summarized and presented in the form of a timeline

From Flies to Robots: Inverted Landing in Small Quadcopters with Dynamic Perching

Inverted landing is a routine behavior among a number of animal fliers. However, mastering this feat poses a considerable challenge for robotic fliers, especially to perform dynamic perching with rapid body rotations (or flips) and landing against gravity. Inverted landing in flies have suggested that optical flow senses are closely linked to the precise triggering and control of body flips that lead to a variety of successful landing behaviors. Building upon this knowledge, we aimed to replicate the flies' landing behaviors in small quadcopters by developing a control policy general to arbitrary ceiling-approach conditions. First, we employed reinforcement learning in simulation to optimize discrete sensory-motor pairs across a broad spectrum of ceiling-approach velocities and directions. Next, we converted the sensory-motor pairs to a two-stage control policy in a continuous augmented-optical flow space. The control policy consists of a first-stage Flip-Trigger Policy, which employs a one-class support vector machine, and a second-stage Flip-Action Policy, implemented as a feed-forward neural network. To transfer the inverted-landing policy to physical systems, we utilized domain randomization and system identification techniques for a zero-shot sim-to-real transfer. As a result, we successfully achieved a range of robust inverted-landing behaviors in small quadcopters, emulating those observed in flies.Comment: 17 pages, 19 Figures, Journal paper currently under revie

Robust Control of Nonlinear Systems with applications to Aerial Manipulation and Self Driving Cars

This work considers the problem of planning and control of robots in an environment with obstacles and external disturbances. The safety of robots is harder to achieve when planning in such uncertain environments. We describe a robust control scheme that combines three key components: system identification, uncertainty propagation, and trajectory optimization. Using this control scheme we tackle three problems. First, we develop a Nonlinear Model Predictive Controller (NMPC) for articulated rigid bodies and apply it to an aerial manipulation system to grasp object mid-air. Next, we tackle the problem of obstacle avoidance under unknown external disturbances. We propose two approaches, the first approach using adaptive NMPC with open- loop uncertainty propagation and the second approach using Tube NMPC. After that, we introduce dynamic models which use Artificial Neural Networks (ANN) and combine them with NMPC to control a ground vehicle and an aerial manipulation system. Finally, we introduce a software framework for integrating the above algorithms to perform complex tasks. The software framework provides users with the ability to design systems that are robust to control and hardware failures where preventive action is taken before-hand. The framework also allows for safe testing of control and task logic in simulation before evaluating on the real robot. The software framework is applied to an aerial manipulation system to perform a package sorting task, and extensive experiments demonstrate the ability of the system to recover from failures. In addition to robust control, we present two related control problems. The first problem pertains to designing an obstacle avoidance controller for an underactuated system that is Lyapunov stable. We extend a standard gyroscopic obstacle avoidance controller to be applicable to an underactuated system. The second problem addresses the navigation of an Unmanned Ground Vehicle (UGV) on an unstructured terrain. We propose using NMPC combined with a high fidelity physics engine to generate a reference trajectory that is dynamically feasible and accounts for unsafe areas in the terrain

Multi-agent Collision Avoidance Using Interval Analysis and Symbolic Modelling with its Application to the Novel Polycopter

Coordination is fundamental component of autonomy when a system is defined by multiple mobile agents. For unmanned aerial systems (UAS), challenges originate from their low-level systems, such as their flight dynamics, which are often complex. The thesis begins by examining these low-level dynamics in an analysis of several well known UAS using a novel symbolic component-based framework. It is shown how this approach is used effectively to define key model and performance properties necessary of UAS trajectory control. This is demonstrated initially under the context of linear quadratic regulation (LQR) and model predictive control (MPC) of a quadcopter. The symbolic framework is later extended in the proposal of a novel UAS platform, referred to as the ``Polycopter" for its morphing nature. This dual-tilt axis system has unique authority over is thrust vector, in addition to an ability to actively augment its stability and aerodynamic characteristics. This presents several opportunities in exploitative control design. With an approach to low-level UAS modelling and control proposed, the focus of the thesis shifts to investigate the challenges associated with local trajectory generation for the purpose of multi-agent collision avoidance. This begins with a novel survey of the state-of-the-art geometric approaches with respect to performance, scalability and tolerance to uncertainty. From this survey, the interval avoidance (IA) method is proposed, to incorporate trajectory uncertainty in the geometric derivation of escape trajectories. The method is shown to be more effective in ensuring safe separation in several of the presented conditions, however performance is shown to deteriorate in denser conflicts. Finally, it is shown how by re-framing the IA problem, three dimensional (3D) collision avoidance is achieved. The novel 3D IA method is shown to out perform the original method in three conflict cases by maintaining separation under the effects of uncertainty and in scenarios with multiple obstacles. The performance, scalability and uncertainty tolerance of each presented method is then examined in a set of scenarios resembling typical coordinated UAS operations in an exhaustive Monte-Carlo analysis