Hierarchical Collision Avoidance for Adaptive-Speed Multirotor Teleoperation

This paper improves safe motion primitives-based teleoperation of a multirotor by developing a hierarchical collision avoidance method that modulates maximum speed based on environment complexity and perceptual constraints. Safe speed modulation is challenging in environments that exhibit varying clutter. Existing methods fix maximum speed and map resolution, which prevents vehicles from accessing tight spaces and places the cognitive load for changing speed on the operator. We address these gaps by proposing a high-rate (10 Hz) teleoperation approach that modulates the maximum vehicle speed through hierarchical collision checking. The hierarchical collision checker simultaneously adapts the local map's voxel size and maximum vehicle speed to ensure motion planning safety. The proposed methodology is evaluated in simulation and real-world experiments and compared to a non-adaptive motion primitives-based teleoperation approach. The results demonstrate the advantages of the proposed teleoperation approach both in time taken and the ability to complete the task without requiring the user to specify a maximum vehicle speed.Comment: 8 pages, 8 figures, to be published in the 2022 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR

Onboard Robust Nonlinear Control for Multiple Multirotor UAVs

In this thesis, we focus on developing and validating onboard robust nonlinear control approaches for multiple multirotor unmanned aerial vehicles (UAVs), for the promise of achieving nontrivial tasks, such as path following with aggressive maneuvers, navigation in complex environments with obstacles, and formation in group. To fulfill these challenging missions, the first concern comes with the stability of flight control for the aggressive UAV maneuvers with large tilted angles. In addition, robustness of control is highly desired in order to lead the multirotor UAVs to safe and accurate performance under disturbances. Furthermore, efficiency of control algorithm is a crucial element for the onboard implementation with limited computational capability. Finally, the potential to simultaneously control a group of UAVs in a stable fashion is required. All of these concerns motivate our work in this thesis in the following aspects. We first propose the problem of robust control for the nontrivial maneuvers of a multirotor UAV under disturbances. A complete framework is developed to enable the UAV to achieve the challenging tasks, which consists of a nonlinear attitude controller based on the solution of global output regulation problems for the rigid body rotations SO(3), a backstepping-like position controller, a six-dimensional (6D) wrench observer to estimate the unknown force and torque disturbances, and an online trajectory planner based on a model predictive control (MPC) method. We prove the strong convergence properties of the proposed method both in theory and via intensive real-robot experiments of aggressive waypoint navigation and large-tilted path following tasks in the presence of external disturbances, e.g. wind gusts. Secondly, we propose the problem of autonomous navigation of a multirotor UAV in complex scenarios. We present an effective and robust control approach, namely a fast MPC method with the inclusion of nonlinear obstacle avoiding constraints, and implement it onboard the UAV at 50Hz. The developed approach enables the navigation for a multirotor UAV in 3D environments with multiple obstacles, by autonomously deciding to fly over or around the randomly located obstacles. The third problem that is addressed in our work is formation control for a group of multirotor UAVs. We solve this problem by proposing a distributed formation control algorithm for multiple UAVs based on the solution of retraction balancing problem. The algorithm brings the whole group of UAVs simultaneously to a prescribed submanifold that determines the formation shape in an asymptotically stable fashion in 2D and 3D environments. We validate our proposed algorithm via a series of hardware-in-the-loop simulations and real-robot experiments in various formation cases of arbitrary time-varying (e.g. expanding, shrinking or moving) shapes. In the actual experiments, up to 4 multirotors have been implemented to form arbitrary triangular, rectangular and circular shapes drawn by the operator via a human-robot-interaction device. We have also carried out virtual tests using up to 6 onboard computers to achieve a spherical formation and a formation moving through obstacles.In dieser Arbeit konzentrieren wir uns auf die Entwicklung und Validierung von robusten nichtlinearen On-Bord Steuerungsansatzen für mehrere unbemannte Multirotor-Luftfahrzeuge (UAVs), mit dem Ziel, nicht triviale Aufgaben zu erledigen wie z.B. Wegfolge mit aggressiven Manovern, Navigation in komplexen Umgebungen mit Hindernissen und Formationsflug in einer Gruppe. Um diese anspruchsvollen Missionen zu erfullen liegt unser Hauptaugenmerk bei der Stabilität der Flugsteuerung für aggressive UAV Manöver mit steilen Lagewinkeln. Des weiteren ist Kontroll-robustheit sehr wünschenswert, um die Multirotor-UAVs unter Beeinflussung sicher und genau zu steuern. Daruber hinaus ist die Effizienz des Kontrollalgorithmus ein wichtiges Element für die Onboard-Implementierung mit eingeschrankter Rechenfähigkeit. Abschliessend ist das Potenzial, gleichzeitig eine Gruppe von UAVs in stabiler Weise zu kontrollieren, erforderlich. All dies motiviert uns zur Arbeit an den folgenden Aspekten: Zuerst behandeln wir das Problem der robusten Steuerung nichttrivialer Manöver eines Multirotor UAV unter Störeinfluss. Ein komplettes Framework wird entwickelt, welches dem UAV ermöglicht diese anspruchsvollen Aufgaben zu bewältigen. Es beinhaltet einem nichtlinearen Lageregler, basierend auf der Lösung von globalen Ausgangsrege lungsproblemen für Starrkörperrotationen SO(3), einem backstepping basierten Positionsregler, einen sechsdimensionalen (6D) wrench observer um die unbekannten Kraftund Drehmomenteinflusse zu schätzen, sowie einem Online-Trajektorienplaner basierend auf Model Predictive Control (MPC). Wir weisen die starken Konvergenzcharakteristiken der vorgeschlagenen Methode nach, sowohl in der Theorie als auchmittels intensiver Real-roboter-Experimente, mit aggressiver Wegpunktnavigation und Wegfindungsaufgaben in extremer Fluglage in Gegenwart externer Einflüsse, z.B. Windböen. Als nächstes bearbeiten wir das Problem der autonomen Navigation eines Multirotor UAV in komplexen Szenarien. Wir stellen einen effektiven und robusten Steuerungsansatz dar, nämlich eine schnelle MPC-Methode mit der Einbeziehung von nichtlinearer Einschränkungen zur Hindernisvermeidung, und implmenetieren diese an Bord des UAV mit 50Hz. Der entwickelte Ansatz ermöglicht die Navigation eines Multirotor UAVs in 3D-Umgebungen mit mehreren Hindernissen, wobei autonom entschieden wir, über oder um die zufällig gelegenen Hindernisse zu fliegen. Das dritte Problem, das in unserer Arbeit angesprochen wird, ist die Bildungssteuerung für eine Gruppe von Multirotor UAVs. Wir lösen dieses Problem, indem wir einen verteilten Formationskontrollalgorithmus für mehrere UAVs auf der Grundlage der Lösung des Retraction Balancing Problems vorschlagen. Der Algorithmus bringt die ganze Gruppe von UAVs gleichzeitig auf eine vorgeschriebene Untermanigfaltigkeit, welche die Formation in asymtotisch stabiler Weise in 2D- und 3D-Umgebungen bestimmt. Wir validieren unseren vorgeschlagenen Algorithmus uber eine Reihe von Hardware-in-the- ¨ Loop-Simulationen und Real-Roboter-Experimente mit verschiedenen Formationsvarianten in beliebigen zeitveränderlichen (z. B. expandierenden, schrumpfenden oder bewegten) Formen. In den eigentlichen Experimenten wurden bis zu 4 Multirotoren eingesetzt, um beliebige dreieckige, rechteckige und kreisförmige Formen zu bilden, die vom Bediener über eine Mensch-Roboter-Interaktionsvorrichtung vorgezeichnet wurden. Wir haben auch virtuelle Tests mit bis zu 6 Onboard-Computern durchgeführt, um eine sphärische Formation und eine Formation zu erreichen, die sich durch Hindernisse. bewegt

Aerial Manipulation: A Literature Review

Aerial manipulation aims at combining the versatil- ity and the agility of some aerial platforms with the manipulation capabilities of robotic arms. This letter tries to collect the results reached by the research community so far within the field of aerial manipulation, especially from the technological and control point of view. A brief literature review of general aerial robotics and space manipulation is carried out as well

Aerial Robotics for Inspection and Maintenance

Aerial robots with perception, navigation, and manipulation capabilities are extending the range of applications of drones, allowing the integration of different sensor devices and robotic manipulators to perform inspection and maintenance operations on infrastructures such as power lines, bridges, viaducts, or walls, involving typically physical interactions on flight. New research and technological challenges arise from applications demanding the benefits of aerial robots, particularly in outdoor environments. This book collects eleven papers from different research groups from Spain, Croatia, Italy, Japan, the USA, the Netherlands, and Denmark, focused on the design, development, and experimental validation of methods and technologies for inspection and maintenance using aerial robots

Compliant aerial manipulation.

The aerial manipulation is a research field which proposes the integration of robotic manipulators in aerial platforms, typically multirotors – widely known as “drones” – or autonomous helicopters. The development of this technology is motivated by the convenience to reduce the time, cost and risk associated to the execution of certain operations or tasks in high altitude areas or difficult access workspaces. Some illustrative application examples are the detection and insulation of leaks in pipe structures in chemical plants, repairing the corrosion in the blades of wind turbines, the maintenance of power lines, or the installation and retrieval of sensor devices in polluted areas. Although nowadays it is possible to find a wide variety of commercial multirotor platforms with payloads from a few gramps up to several kilograms, and flight times around thirty minutes, the development of an aerial manipulator is still a technological challenge due to the strong requirements relative to the design of the manipulator in terms of very low weight, low inertia, dexterity, mechanical robustness and control. The main contribution of this thesis is the design, development and experimental validation of several prototypes of lightweight (<2 kg) and compliant manipulators to be integrated in multirotor platforms, including human-size dual arm systems, compliant joint arms equipped with human-like finger modules for grasping, and long reach aerial manipulators. Since it is expected that the aerial manipulator is capable to execute inspection and maintenance tasks in a similar way a human operator would do, this thesis proposes a bioinspired design approach, trying to replicate the human arm in terms of size, kinematics, mass distribution, and compliance. This last feature is actually one of the key concepts developed and exploited in this work. Introducing a flexible element such as springs or elastomers between the servos and the links extends the capabilities of the manipulator, allowing the estimation and control of the torque/force, the detection of impacts and overloads, or the localization of obstacles by contact. It also improves safety and efficiency of the manipulator, especially during the operation on flight or in grabbing situations, where the impacts and contact forces may damage the manipulator or destabilize the aerial platform. Unlike most industrial manipulators, where force-torque control is possible at control rates above 1 kHz, the servo actuators typically employed in the development of aerial manipulators present important technological limitations: no torque feedback nor control, only position (and in some models, speed) references, low update rates (<100 Hz), and communication delays. However, these devices are still the best solution due to their high torque to weight ratio, low cost, compact design, and easy assembly and integration. In order to cope with these limitations, the compliant joint arms presented here estimate and control the wrenches from the deflection of the spring-lever transmission mechanism introduced in the joints, measured at joint level with encoders or potentiometers, or in the Cartesian space employing vision sensors. Note that in the developed prototypes, the maximum joint deflection is around 25 degrees, which corresponds to a deviation in the position of the end effector around 20 cm for a human-size arm. The capabilities and functionalities of the manipulators have been evaluated in fixed base test-bench firstly, and then in outdoor flight tests, integrating the arms in different commercial hexarotor platforms. Frequency characterization, position/force/impedance control, bimanual grasping, arm teleoperation, payload mass estimation, or contact-based obstacle localization are some of the experiments presented in this thesis that validate the developed prototypes.La manipulación aérea es un campo de investigación que propone la integración de manipuladores robóticos in plataformas aéreas, típicamente multirotores – comúnmente conocidos como “drones” – o helicópteros autónomos. El desarrollo de esta tecnología está motivada por la conveniencia de reducir el tiempo, coste y riesgo asociado a la ejecución de ciertas operaciones o tareas en áreas de gran altura o espacios de trabajo de difícil acceso. Algunos ejemplos ilustrativos de aplicaciones son la detección y aislamiento de fugas en estructura de tuberías en plantas químicas, la reparación de la corrosión en las palas de aerogeneradores, el mantenimiento de líneas eléctricas, o la instalación y recuperación de sensores en zonas contaminadas. Aunque hoy en día es posible encontrar una amplia variedad de plataformas multirotor comerciales con cargas de pago desde unos pocos gramos hasta varios kilogramos, y tiempo de vuelo entorno a treinta minutos, el desarrollo de los manipuladores aéreos es todavía un desafío tecnológico debido a los exigentes requisitos relativos al diseño del manipulador en términos de muy bajo peso, baja inercia, destreza, robustez mecánica y control. La contribución principal de esta tesis es el diseño, desarrollo y validación experimental de varios prototipos de manipuladores de bajo peso (<2 kg) con capacidad de acomodación (“compliant”) para su integración en plataformas aéreas multirotor, incluyendo sistemas bi-brazo de tamaño humano, brazos robóticos de articulaciones flexibles con dedos antropomórficos para agarre, y manipuladores aéreos de largo alcance. Puesto que se prevé que el manipulador aéreo sea capaz de ejecutar tareas de inspección y mantenimiento de forma similar a como lo haría un operador humano, esta tesis propone un enfoque de diseño bio-inspirado, tratando de replicar el brazo humano en cuanto a tamaño, cinemática, distribución de masas y flexibilidad. Esta característica es de hecho uno de los conceptos clave desarrollados y utilizados en este trabajo. Al introducir un elemento elástico como los muelles o elastómeros entre el los actuadores y los enlaces se aumenta las capacidades del manipulador, permitiendo la estimación y control de las fuerzas y pares, la detección de impactos y sobrecargas, o la localización de obstáculos por contacto. Además mejora la seguridad y eficiencia del manipulador, especialmente durante las operaciones en vuelo, donde los impactos y fuerzas de contacto pueden dañar el manipulador o desestabilizar la plataforma aérea. A diferencia de la mayoría de manipuladores industriales, donde el control de fuerzas y pares es posible a tasas por encima de 1 kHz, los servo motores típicamente utilizados en el desarrollo de manipuladores aéreos presentan importantes limitaciones tecnológicas: no hay realimentación ni control de torque, sólo admiten referencias de posición (o bien de velocidad), y presentan retrasos de comunicación. Sin embargo, estos dispositivos son todavía la mejor solución debido al alto ratio de torque a peso, por su bajo peso, diseño compacto y facilidad de ensamblado e integración. Para suplir estas limitaciones, los brazos robóticos flexibles presentados aquí permiten estimar y controlar las fuerzas a partir de la deflexión del mecanismo de muelle-palanca introducido en las articulaciones, medida a nivel articular mediante potenciómetros o codificadores, o en espacio Cartesiano mediante sensores de visión. Tómese como referencia que en los prototipos desarrollados la máxima deflexión articular es de unos 25 grados, lo que corresponde a una desviación de posición en torno a 20 cm en el efector final para un brazo de tamaño humano. Las capacidades y funcionalidades de estos manipuladores se han evaluado en base fija primero, y luego en vuelos en exteriores, integrando los brazos en diferentes plataformas hexartor comerciales. Caracterización frecuencial, control de posición/fuerza/impedancia, agarre bimanual, teleoperación de brazos, estimación de carga, o la localización de obstáculos mediante contacto son algunos de los experimentos presentados en esta tesis para validar los prototipos desarrollados por el auto

Leonardo Drone Contest Autonomous Drone Competition: Overview, Results, and Lessons Learned from Politecnico di Milano Team

In this paper, the Politecnico di Milano solutions proposed for the Leonardo Drone Contest (LDC) are presented. The Leonardo Drone Contest is an annual autonomous drone competition among universities, which has already seen the conclusion of its second edition. In each edition, the participating teams were asked to design and build an autonomous multicopter, capable of accomplishing complex tasks in an indoor urban-like environment. To reach this goal, the designed systems should be capable of navigating in a Global Navigation Satellite System (GNSS)-denied environment with autonomous decision making, online planning and collision avoidance capabilities. In this light, the authors describe the first two editions of the competition, i.e., their rules, objectives and overview of the proposed solutions. While the first edition is presented as relevant for the experience and takeaways acquired from it, the second edition solution is analyzed in detail, providing both the simulation and experimental results obtained

Invariant Set Distributed Explicit Reference Governors for Provably Safe On-Board Control of Nano-Quadrotor Swarms

This article provides a theory for provably safe and computationally efficient distributed constrained control, and describes an application to a swarm of nano-quadrotors with limited on-board hardware and subject to multiple state and input constraints. We provide a formal extension of the explicit reference governor framework to address the case of distributed systems. The efficacy, robustness, and scalability of the proposed theory is demonstrated by an extensive experimental validation campaign and a comparative simulation study on single and multiple nano-quadrotors. The control strategy is implemented in real-time on-board palm-sized unmanned erial vehicles, and achieves safe swarm coordination without relying on any offline trajectory computations

Doctor of Philosophy

dissertationThis dissertation solves the collision avoidance problem for single- and multi-robot systems where dynamic effects are significant. In many robotic systems (e.g., highly maneuverable and agile unmanned aerial vehicles) the dynamics cannot be ignored and collision avoidance schemes based on kinematic models can result in collisions or provide limited performance, especially at high operating speeds. Herein, real-time, model-based collision avoidance algorithms that explicitly consider the robots' dynamics and perform real-time input changes to alter the trajectory and steer the robot away from potential collisions are developed, implemented, and verified in simulations and physical experiments. Such algorithms are critical in applications where a high degree of autonomy and performance are needed, for example in robot-assisted first response where aerial and/or mobile ground robots are required to maneuver quickly through cluttered and dangerous environments in search of survivors. Firstly, the research extends reciprocal collision avoidance to robots with dynamics by unifying previous approaches to reciprocal collision avoidance under a single, generalized representation using control obstacles. In fact, it is shown how velocity obstacles, acceleration velocity obstacles, continuous control obstacles, and linear quadratic regulator (LQR)-obstacles are special instances of the generalized framework. Furthermore, an extension of control obstacles to general reciprocal collision avoidance for nonlinear, nonhomogeneous systems where the robots may have different state spaces and different nonlinear equations of motion from one another is described. Both simulations and physical experiments are provided for a combination of differential-drive, differential-drive with a trailer, and car-like robots to demonstrate that the approach is capable of letting a nonhomogeneous group of robots with nonlinear equations of motion safely avoid collisions at real-time computation rates. Secondly, the research develops a stochastic collision avoidance algorithm for a tele-operated unmanned aerial vehicle (UAV) that considers uncertainty in the robot's dynamics model and the obstacles' position as measured from sensors. The model-based automatic collision avoidance algorithm is implemented on a custom-designed quadcopter UAV system with on-board computation and the sensor data are processed using a split-and-merge segmentation algorithm and an approximate Minkowski difference. Flight tests are conducted to validate the algorithm's capabilities for providing tele-operated collision-free operation. Finally, a set of human subject studies are performed to quantitatively compare the performance between the model-based algorithm, the basic risk field algorithm (a variant on potential field), and full manual control. The results show that the model-based algorithm performs significantly better than manual control in both the number of collisions and the UAV's average speed, both of which are extremely vital, for example, for UAV-assisted search and rescue applications. Compared to the potential-field-based algorithm, the model-based algorithm allowed the pilot to operate the UAV with higher average speeds