System Architectures for Cooperative Teams of Unmanned Aerial Vehicles Interacting Physically with the Environment

Unmanned Aerial Vehicles (UAVs) have become quite a useful tool for a wide range of applications, from inspection & maintenance to search & rescue, among others. The capabilities of a single UAV can be extended or complemented by the deployment of more UAVs, so multi-UAV cooperative teams are becoming a trend. In that case, as di erent autopilots, heterogeneous platforms, and application-dependent software components have to be integrated, multi-UAV system architectures that are fexible and can adapt to the team's needs are required. In this thesis, we develop system architectures for cooperative teams of UAVs, paying special attention to applications that require physical interaction with the environment, which is typically unstructured. First, we implement some layers to abstract the high-level components from the hardware speci cs. Then we propose increasingly advanced architectures, from a single-UAV hierarchical navigation architecture to an architecture for a cooperative team of heterogeneous UAVs. All this work has been thoroughly tested in both simulation and eld experiments in di erent challenging scenarios through research projects and robotics competitions. Most of the applications required physical interaction with the environment, mainly in unstructured outdoors scenarios. All the know-how and lessons learned throughout the process are shared in this thesis, and all relevant code is publicly available.Los vehículos aéreos no tripulados (UAVs, del inglés Unmanned Aerial Vehicles) se han convertido en herramientas muy valiosas para un amplio espectro de aplicaciones, como inspección y mantenimiento, u operaciones de rescate, entre otras. Las capacidades de un único UAV pueden verse extendidas o complementadas al utilizar varios de estos vehículos simultáneamente, por lo que la tendencia actual es el uso de equipos cooperativos con múltiples UAVs. Para ello, es fundamental la integración de diferentes autopilotos, plataformas heterogéneas, y componentes software -que dependen de la aplicación-, por lo que se requieren arquitecturas multi-UAV que sean flexibles y adaptables a las necesidades del equipo. En esta tesis, se desarrollan arquitecturas para equipos cooperativos de UAVs, prestando una especial atención a aplicaciones que requieran de interacción física con el entorno, cuya naturaleza es típicamente no estructurada. Primero se proponen capas para abstraer a los componentes de alto nivel de las particularidades del hardware. Luego se desarrollan arquitecturas cada vez más avanzadas, desde una arquitectura de navegación para un único UAV, hasta una para un equipo cooperativo de UAVs heterogéneos. Todo el trabajo ha sido minuciosamente probado, tanto en simulación como en experimentos reales, en diferentes y complejos escenarios motivados por proyectos de investigación y competiciones de robótica. En la mayoría de las aplicaciones se requería de interacción física con el entorno, que es normalmente un escenario en exteriores no estructurado. A lo largo de la tesis, se comparten todo el conocimiento adquirido y las lecciones aprendidas en el proceso, y el código relevante está publicado como open-source

Optimal Multi-UAV Trajectory Planning for Filming Applications

Teams of multiple Unmanned Aerial Vehicles (UAVs) can be used to record large-scale outdoor scenarios and complementary views of several action points as a promising system for cinematic video recording. Generating the trajectories of the UAVs plays a key role, as it should be ensured that they comply with requirements for system dynamics, smoothness, and safety. The rise of numerical methods for nonlinear optimization is finding a ourishing field in optimization-based approaches to multi- UAV trajectory planning. In particular, these methods are rather promising for video recording applications, as they enable multiple constraints and objectives to be formulated, such as trajectory smoothness, compliance with UAV and camera dynamics, avoidance of obstacles and inter-UAV con icts, and mutual UAV visibility. The main objective of this thesis is to plan online trajectories for multi-UAV teams in video applications, formulating novel optimization problems and solving them in real time. The thesis begins by presenting a framework for carrying out autonomous cinematography missions with a team of UAVs. This framework enables media directors to design missions involving different types of shots with one or multiple cameras, running sequentially or concurrently. Second, the thesis proposes a novel non-linear formulation for the challenging problem of computing optimal multi-UAV trajectories for cinematography, integrating UAV dynamics and collision avoidance constraints, together with cinematographic aspects such as smoothness, gimbal mechanical limits, and mutual camera visibility. Lastly, the thesis describes a method for autonomous aerial recording with distributed lighting by a team of UAVs. The multi-UAV trajectory optimization problem is decoupled into two steps in order to tackle non-linear cinematographic aspects and obstacle avoidance at separate stages. This allows the trajectory planner to perform in real time and to react online to changes in dynamic environments. It is important to note that all the methods in the thesis have been validated by means of extensive simulations and field experiments. Moreover, all the software components have been developed as open source.Los equipos de vehículos aéreos no tripulados (UAV) son sistemas prometedores para grabar eventos cinematográficos, en escenarios exteriores de grandes dimensiones difíciles de cubrir o para tomar vistas complementarias de diferentes puntos de acción. La generación de trayectorias para este tipo de vehículos desempeña un papel fundamental, ya que debe garantizarse que se cumplan requisitos dinámicos, de suavidad y de seguridad. Los enfoques basados en la optimización para la planificación de trayectorias de múltiples UAVs se pueden ver beneficiados por el auge de los métodos numéricos para la resolución de problemas de optimización no lineales. En particular, estos métodos son bastante prometedores para las aplicaciones de grabación de vídeo, ya que permiten formular múltiples restricciones y objetivos, como la suavidad de la trayectoria, el cumplimiento de la dinámica del UAV y de la cámara, la evitación de obstáculos y de conflictos entre UAVs, y la visibilidad mutua. El objetivo principal de esta tesis es planificar trayectorias para equipos multi-UAV en aplicaciones de vídeo, formulando novedosos problemas de optimización y resolviéndolos en tiempo real. La tesis comienza presentando un marco de trabajo para la realización de misiones cinematográficas autónomas con un equipo de UAVs. Este marco permite a los directores de medios de comunicación diseñar misiones que incluyan diferentes tipos de tomas con una o varias cámaras, ejecutadas de forma secuencial o concurrente. En segundo lugar, la tesis propone una novedosa formulación no lineal para el difícil problema de calcular las trayectorias óptimas de los vehículos aéreos no tripulados en cinematografía, integrando en el problema la dinámica de los UAVs y las restricciones para evitar colisiones, junto con aspectos cinematográficos como la suavidad, los límites mecánicos del cardán y la visibilidad mutua de las cámaras. Por último, la tesis describe un método de grabación aérea autónoma con iluminación distribuida por un equipo de UAVs. El problema de optimización de trayectorias se desacopla en dos pasos para abordar los aspectos cinematográficos no lineales y la evitación de obstáculos en etapas separadas. Esto permite al planificador de trayectorias actuar en tiempo real y reaccionar en línea a los cambios en los entornos dinámicos. Es importante señalar que todos los métodos de la tesis han sido validados mediante extensas simulaciones y experimentos de campo. Además, todos los componentes del software se han desarrollado como código abierto

Control and communication systems for automated vehicles cooperation and coordination

Mención Internacional en el título de doctorThe technological advances in the Intelligent Transportation Systems (ITS) are exponentially improving over the last century. The objective is to provide intelligent and innovative services for the different modes of transportation, towards a better, safer, coordinated and smarter transport networks. The Intelligent Transportation Systems (ITS) focus is divided into two main categories; the first is to improve existing components of the transport networks, while the second is to develop intelligent vehicles which facilitate the transportation process. Different research efforts have been exerted to tackle various aspects in the fields of the automated vehicles. Accordingly, this thesis is addressing the problem of multiple automated vehicles cooperation and coordination. At first, 3DCoAutoSim driving simulator was developed in Unity game engine and connected to Robot Operating System (ROS) framework and Simulation of Urban Mobility (SUMO). 3DCoAutoSim is an abbreviation for "3D Simulator for Cooperative Advanced Driver Assistance Systems (ADAS) and Automated Vehicles Simulator". 3DCoAutoSim was tested under different circumstances and conditions, afterward, it was validated through carrying-out several controlled experiments and compare the results against their counter reality experiments. The obtained results showed the efficiency of the simulator to handle different situations, emulating real world vehicles. Next is the development of the iCab platforms, which is an abbreviation for "Intelligent Campus Automobile". The platforms are two electric golf-carts that were modified mechanically, electronically and electrically towards the goal of automated driving. Each iCab was equipped with several on-board embedded computers, perception sensors and auxiliary devices, in order to execute the necessary actions for self-driving. Moreover, the platforms are capable of several Vehicle-to-Everything (V2X) communication schemes, applying three layers of control, utilizing cooperation architecture for platooning, executing localization systems, mapping systems, perception systems, and finally several planning systems. Hundreds of experiments were carried-out for the validation of each system in the iCab platform. Results proved the functionality of the platform to self-drive from one point to another with minimal human intervention.Los avances tecnológicos en Sistemas Inteligentes de Transporte (ITS) han crecido de forma exponencial durante el último siglo. El objetivo de estos avances es el de proveer de sistemas innovadores e inteligentes para ser aplicados a los diferentes medios de transporte, con el fin de conseguir un transporte mas eficiente, seguro, coordinado e inteligente. El foco de los ITS se divide principalmente en dos categorías; la primera es la mejora de los componentes ya existentes en las redes de transporte, mientras que la segunda es la de desarrollar vehículos inteligentes que hagan más fácil y eficiente el transporte. Diferentes esfuerzos de investigación se han llevado a cabo con el fin de solucionar los numerosos aspectos asociados con la conducción autónoma. Esta tesis propone una solución para la cooperación y coordinación de múltiples vehículos. Para ello, en primer lugar se desarrolló un simulador (3DCoAutoSim) de conducción basado en el motor de juegos Unity, conectado al framework Robot Operating System (ROS) y al simulador Simulation of Urban Mobility (SUMO). 3DCoAutoSim ha sido probado en diferentes condiciones y circunstancias, para posteriormente validarlo con resultados a través de varios experimentos reales controlados. Los resultados obtenidos mostraron la eficiencia del simulador para manejar diferentes situaciones, emulando los vehículos en el mundo real. En segundo lugar, se desarrolló la plataforma de investigación Intelligent Campus Automobile (iCab), que consiste en dos carritos eléctricos de golf, que fueron modificados eléctrica, mecánica y electrónicamente para darle capacidades autónomas. Cada iCab se equipó con diferentes computadoras embebidas, sensores de percepción y unidades auxiliares, con la finalidad de transformarlos en vehículos autónomos. Además, se les han dado capacidad de comunicación multimodal (V2X), se les han aplicado tres capas de control, incorporando una arquitectura de cooperación para operación en modo tren, diferentes esquemas de localización, mapeado, percepción y planificación de rutas. Innumerables experimentos han sido realizados para validar cada uno de los diferentes sistemas incorporados. Los resultados prueban la funcionalidad de esta plataforma para realizar conducción autónoma y cooperativa con mínima intervención humana.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Francisco Javier Otamendi Fernández de la Puebla.- Secretario: Hanno Hildmann.- Vocal: Pietro Cerr

UNMANNED GROUND VEHICLE (UGV) DOCKING, CONNECTION, AND CABLING FOR ELECTRICAL POWER TRANSMISSION IN AUTONOMOUS MOBILE MICROGRIDS

Autonomous Mobile Microgrids provide electrical power to loads in environments where humans either can not, or would prefer not to, perform the task of positioning and connecting the power grid equipment. The contributions of this work compose an architecture for electrical power transmission by Unmanned Ground Vehicles (UGV). Purpose-specific UGV docking and cable deployment software algorithms, and hardware for electrical connection and cable management, has been deployed on Clearpath Husky robots. Software development leverages Robot Operating System (ROS) tools for navigation and rendezvous of the autonomous UGV robots, with task-specific visual feedback controllers for docking validated in Monte-Carlo outdoor trials with a 73% docking rate, and application to wireless power transmission demonstrated in an outdoor environment. An “Adjustable Cable Management Mechanism” (ACMM) was designed to meet low cost, compact-platform constraints for powered deployment and retraction by a UGV of electrical cable subject to disturbance, with feed rates up to 1 m/s. A probe-and-funnel AC/DC electrical connector system was de- veloped for deployment on UGVs, which does not substantially increase the cost or complexity of the UGV, while providing a repeatable and secure method of coupling electrical contacts subject to a docking miss-alignment of up to +/-2 cm laterally and +/-15 degrees axially. Cabled power transmission is accomplished by a feed-forward/feedback control method, which utilizes visual estimation of the cable state to deploy electrical cable without tension, in the obstacle-free track of the UGV as it transverses to connect power grid nodes. Cabling control response to step-input UGV chassis velocities in the forward, reverse, and zero-point-turn maneuvers are presented, as well as outdoor cable deployment. This power transmission capability is relevant to diverse domains including military Forward-Operating-Bases, disaster response, robotic persistent operation, underwater mining, or planetary exploration

Cooperation and Autonomy for UAV Swarms

In the last few years, the level of autonomy of mini- and micro-Unmanned Aerial Vehicles (UAVs) has increased thanks to the miniaturization of flight control systems and payloads, and the availability of computationally affordable algorithms for autonomous Guidance Navigation and Control (GNC). However, despite the technological evolution, operations conducted by a single micro-UAV still present limits in terms of performance, coverage and reliability. The scope of this thesis is to overcome single-UAV limits by developing new distributed GNC architectures and technologies where the cooperative nature of a UAV formation is exploited to obtain navigation information. Moreover, this thesis aims at increasing UAVs autonomy by developing a take-off and landing technique which permits to complete fully autonomous operations, also taking into account regulations and the required level of safety. Indeed, in addition to the typical performance limitations of micro-UAVs, this thesis takes into account also those applications where a multi-vehicle architecture can improve coverage and reliability, and allow real time data fusion. Furthermore, considering the low cost of micro-UAV systems with consumer grade avionics, having several UAVs can be more cost effective than equipping a single vehicle with high performance equipment. Among several research challenges to be addressed in order to design and operate a distributed system of vehicles working together for real time applications, this thesis focuses on the following topics regarding cooperation and autonomy: Improvement of UAV navigation performance: This research topic aims at improving the navigation performance of an UAV flying cooperatively with one or more UAVs, considering that the only integration of low cost inertial measurement units (IMUs), Global Navigation Satellite Systems (GNSS) and magnetometers allows real time stabilization and flight control but may not be suitable for applications requiring fine sensor pointing. The focus is set on outdoor environments and it is assumed that all vehicles of the formation are flying under nominal Global Positioning System (GPS) coverage, hence, the main navigation improvement is in terms of attitude estimation. In particular, the key concept is to exploit Differential GPS (DGPS) among vehicles and vision-based tracking to build a virtual additional navigation sensor whose information is then integrated within a sensor fusion algorithm based on an Extended Kalman Filter (EKF). Both numerical simulations and flight results show the potential of sub-degree angular accuracy. In particular, proper formation geometries, and even relatively small baselines, allow achieving a heading uncertainty that can approach 0.1°, which represents a very important result taking into account typical performance levels of IMUs onboard small UAVs. UAV navigation in GPS challenging environments: This research topic aims at developing algorithms for improving navigation performance of UAVs flying in GPS-challenging environments (e.g. natural or urban canyons, or mixed outdoor-indoor settings), where GPS measurements can be unavailable and/or unreliable. These algorithms exploit aiding measurements from one or more cooperative UAVs flying under nominal GPS coverage and are based on the concepts of relative sensing and information sharing. The developed sensor fusion architecture is based on a tightly coupled EKF that integrates measurements from onboard inertial sensors and magnetometers, the available GPS pseudoranges, position information from cooperative UAVs, and line-of-sight information derived by visual sensors. In addition, if available, measurements coming from a monocular pose estimation algorithm can be integrated within the developed EKF in order to counteract the position error drift. Results show that aiding measurements from a single cooperative UAV do not allow eliminating position error drift. However, combining this approach with a standalone visual-SLAM, integrating valid pseudoranges in the tightly coupled filtering structure, or exploiting ad hoc commanded motion of the cooperative vehicle under GPS coverage drastically reduces the position error drift keeping meter-level positioning accuracy also in absence of reliable GPS observables. Autonomous take-off and landing: This research activity, conducted during a 6 month Academic Guest period at ETH Zürich, focuses on increasing reliability, versatility and flight time of UAVs, by developing an autonomous take-off and landing technique. Often, the landing phase is the most critical as it involves performing delicate maneuvers; e.g., landing on a station for recharging or on a ground carrier for transportation. These procedures are subject to constraints on time and space and must be robust to changes in environmental conditions. These problems are addressed in this thesis, where a guidance approach, based on the intrinsic Tau guidance theory, is integrated within the end-to-end software developed at ETH Zürich. This method has been validated both in simulations and through real platform experiments by using rotary-wing UAVs to land on static platforms. Results show that this method achieves smooth landings within 10 cm accuracy, with easily adjustable trajectory parameters

Advances in Human Robot Interaction for Cloud Robotics applications

In this thesis are analyzed different and innovative techniques for Human Robot Interaction. The focus of this thesis is on the interaction with flying robots. The first part is a preliminary description of the state of the art interactions techniques. Then the first project is Fly4SmartCity, where it is analyzed the interaction between humans (the citizen and the operator) and drones mediated by a cloud robotics platform. Then there is an application of the sliding autonomy paradigm and the analysis of different degrees of autonomy supported by a cloud robotics platform. The last part is dedicated to the most innovative technique for human-drone interaction in the User’s Flying Organizer project (UFO project). This project wants to develop a flying robot able to project information into the environment exploiting concepts of Spatial Augmented Realit

Enabling Multi-LiDAR Sensing in GNSS-Denied Environments: SLAM Dataset, Benchmark, and UAV Tracking with LiDAR-as-a-camera

The rise of Light Detection and Ranging (LiDAR) sensors has profoundly impacted industries ranging from automotive to urban planning. As these sensors become increasingly affordable and compact, their applications are diversifying, driving precision, and innovation. This thesis delves into LiDAR's advancements in autonomous robotic systems, with a focus on its role in simultaneous localization and mapping (SLAM) methodologies and LiDAR as a camera-based tracking for Unmanned Aerial Vehicles (UAV). Our contributions span two primary domains: the Multi-Modal LiDAR SLAM Benchmark, and the LiDAR-as-a-camera UAV Tracking. In the former, we have expanded our previous multi-modal LiDAR dataset by adding more data sequences from various scenarios. In contrast to the previous dataset, we employ different ground truth-generating approaches. We propose a new multi-modal multi-lidar SLAM-assisted and ICP-based sensor fusion method for generating ground truth maps. Additionally, we also supplement our data with new open road sequences with GNSS-RTK. This enriched dataset, supported by high-resolution LiDAR, provides detailed insights through an evaluation of ten configurations, pairing diverse LiDAR sensors with state-of-the-art SLAM algorithms. In the latter contribution, we leverage a custom YOLOv5 model trained on panoramic low-resolution images from LiDAR reflectivity (LiDAR-as-a-camera) to detect UAVs, demonstrating the superiority of this approach over point cloud or image-only methods. Additionally, we evaluated the real-time performance of our approach on the Nvidia Jetson Nano, a popular mobile computing platform. Overall, our research underscores the transformative potential of integrating advanced LiDAR sensors with autonomous robotics. By bridging the gaps between different technological approaches, we pave the way for more versatile and efficient applications in the future