Distributed UAV Swarm Formation and Collision Avoidance Strategies Over Fixed and Switching Topologies

This is the final version. Available on open access from IEEE via the DOI in this recordThis article proposes a controlling framework for multiple unmanned aerial vehicles (UAVs) to integrate the modes of formation flight and swarm deployment over fixed and switching topologies. Formation strategies enable UAVs to enjoy key collective benefits including reduced energy consumption, but the shape of the formation and each UAV's freedom are significantly restrained. Swarm strategies are thus proposed to maximize each UAV's freedom following simple yet powerful rules. This article investigates the integration and switch between these two strategies, considering the deployment environment factors, such as poor network conditions and unknown and often highly mobile obstacles. We design a distributed formation controller to guide multiple UAVs in orderless states to swiftly reach an intended formation. Inspired by starling birds and similar biological creatures, a distributed collision avoidance controller is proposed to avoid unknown and mobile obstacles. We further illustrated the stability of the controllers over both fixed and switching topologies. The experimental results confirm the effectiveness of the framework

Spatial combination of sensor data deriving from mobile platforms for precision farming applications

This thesis combines optical sensors on a ground and on an aerial platform for field measurements in wheat, to identify nitrogen (N) levels, estimating biomass (BM) and predicting yield. The Multiplex Research (MP) fluorescence sensor was used for the first time in wheat. The individual objectives were: (i) Evaluation of different available sensors and sensor platforms used in Precision Farming (PF) to quantify the crop nutrition status, (ii) Acquisition of ground and aerial sensor data with two ground spectrometers, an aerial spectrometer and a ground fluorescence sensor, (iii) Development of effective post-processing methods for correction of the sensor data, (iv) Analysis and evaluation of the sensors with regard to the mapping of biomass, yield and nitrogen content in the plant, and (v) Yield simulation as a function of different sensor signals. This thesis contains three papers, published in international peer-reviewed journals. The first publication is a literature review on sensor platforms used in agricultural research. A subdivision of sensors and their applications was done, based on a detailed categorization model. It evaluates strengths and weaknesses, and discusses research results gathered with aerial and ground platforms with different sensors. Also, autonomous robots and swarm technologies suitable for PF tasks were reviewed. The second publication focuses on spectral and fluorescence sensors for BM, yield and N detection. The ground sensors were mounted on the Hohenheim research sensor platform Sensicle. A further spectrometer was installed in a fixed-wing Unmanned Aerial Vehicle (UAV). In this study, the sensors of the Sensicle and the UAV were used to determine plant characteristics and yield of three-year field trials at the research station Ihinger Hof, Renningen (Germany), an institution of the University of Hohenheim, Stuttgart (Germany). Winter wheat (Triticum aestivum L.) was sown on three research fields, with different N levels applied to each field. The measurements in the field were geo-referenced and logged with an absolute GPS accuracy of ±2.5 cm. The GPS data of the UAV was corrected based on the pitch and roll position of the UAV at each measurement. In the first step of the data analysis, raw data obtained from the sensors was post-processed and was converted into indices and ratios relating to plant characteristics. The converted ground sensor data were analysed, and the results of the correlations were interpreted related to the dependent variables (DV) BM weight, wheat yield and available N. The results showed significant positive correlations between the DVs and the Sensicle sensor data. For the third paper, the UAV sensor data was included into the evaluations. The UAV data analysis revealed low significant results for only one field in the year 2011. A multirotor UAV was considered as a more viable aerial platform, that allows for more precision and higher payload. Thereby, the ground sensors showed their strength at a close measuring distance to the plant and a smaller measurement footprint. The results of the two ground spectrometers showed significant positive correlations between yield and the indices from CropSpec, NDVI (Normalised Difference Vegetation Index) and REIP (Red-Edge Inflection Point). Also, FERARI and SFR (Simple Fluorescence Ratio) of the MP fluorescence sensor were chosen for the yield prediction model analysis. With the available N, CropSpec and REIP correlated significantly. The BM weight correlated with REIP even at a very early growing stage (Z 31), and with SAVI (Soil-Adjusted Vegetation Index) at ripening stage (Z 85). REIP, FERARI and SFR showed high correlations to the available N, especially in June and July. The ratios and signals of the MP sensor were highly significant compared to the BM weight above Z 85. Both ground spectrometers are suitable for data comparison and data combination with the active MP fluorescence sensor. Through a combination of fluorescence ratios and spectrometer indices, linear models for the prediction of wheat yield were generated, correlating significantly over the course of the vegetative period for research field Lammwirt (LW) in 2012. The best model for field LW in 2012 was selected for cross-validation with the measurements of the fields Inneres Täle (IT) and Riech (RI) in 2011 and 2012. However, it was not significant. By exchanging only one spectral index with a fluorescence ratio in a similar linear model, it showed significant correlations. This work successfully proves the combination of different sensor ratios and indices for the detection of plant characteristics, offering better and more robust predictions and quantifications of field parameters without employing destructive methods. The MP sensor proved to be universally applicable, showing significant correlations to the investigated characteristics such as BM weight, wheat yield and available N.Diese Arbeit kombiniert optische Sensoren auf einer Sensorplattform (SPF) am Boden und in der Luft bei Messungen in Weizen, um die Stickstoff-(N)-Werte zu identifizieren, während gleichzeitig die Biomasse (BM) geschätzt und der Ertrag vorhergesagt wird. Erstmals wurde hierfür der Fluoreszenzsensor Multiplex Research (MP) in Weizen eingesetzt. Die Ziele dieser Dissertation umfassen: (i) Bewertung verfügbarer Sensoren und SPF, die in der Präzisionslandwirtschaft zur Quantifizierung des Ernährungszustandes von Nutzpflanzen verwendet werden, (ii) Erfassung von Daten mit zwei Spektrometern am Boden, einem Spektrometer auf einem Modellflugzeug (UAV) und einem Fluoreszenzsensor am Boden, (iii) Erstellung effektiver Nachbearbeitungsmethoden für die Datenkorrektur, (iv) Analyse und Evaluation der Sensoren für die Abbildung der BM, des Ertrags und des N-Gehaltes in der Pflanze, und (v) Ertragssimulation als Funktion von Merkmalen unterschiedlicher Sensorsignale. Diese Arbeit enthält drei Artikel, die in international begutachteten Fachzeitschriften publiziert wurden. Die erste Veröffentlichung ist eine Literaturrecherche über SPF in der Agrarforschung. Ein detailliertes Kategorisierungsmodell wird für eine allgemeine Unterteilung der Sensoren und deren Anwendungsgebiete herangenommen, die Stärken und Schwächen bewertet, und die Forschungsergebnisse von Luft- und Bodenplattformen mit unterschiedlicher Sensorik diskutiert. Außerdem werden autonome Roboter und für landwirtschaftliche Aufgaben geeignete Schwarmtechnologien beschrieben. Die zweite Publikation fokussiert sich auf Spektral- und Fluoreszenzsensoren für die Erfassung von BM, Ertrag und N. In der Arbeit wurden die Bodensensoren auf der Hohenheimer Forschungs-SPF Sensicle und der Sensor auf dem UAV in dreijährigen Feldversuchen auf der Versuchsstation Ihinger Hof der Universität Hohenheim in Renningen für die Bestimmung von Pflanzenmerkmalen und des Ertrags eingesetzt. Auf drei Versuchsfeldern wurde Winterweizen ausgesät, und in einem randomisierten Versuchsdesign unterschiedliche N-Düngestufen angelegt. Die Sensormessungen im Feld wurden mit einer absoluten GPS Genauigkeit von ±2,5 cm verortet. Die GPS Daten des UAVs wurden mittels der Nick- und Rollposition lagekorrigiert. Im ersten Schritt der Datenanalyse wurden die Sensorrohdaten nachbearbeitet und in Indizes und Ratios umgerechnet. Die Bodensensordaten wurden analysiert, und die Ergebnisse der Korrelationen in Bezug zu den abhängigen Variablen (DV) BM-Gewicht, Weizenertrag, verfügbarer sowie aufgenommener N dargestellt. Die Ergebnisse zeigen signifikant positive Korrelationen zwischen den DVs und den Sensicle-Sensordaten. Für die dritte Publikation wurden die Sensordaten des UAV in die Auswertungen miteinbezogen. Die Analyse der UAV Daten zeigte niedrige signifikante Ergebnisse für nur ein Feld im Versuchsjahr 2011. Ein Multikopter wird als zuverlässigere Luftplattform erachtet, der mehr Präzision und eine höhere Nutzlast ermöglicht. Die Sensoren auf dem Sensicle zeigten ihren Vorteil bedingt durch einen kürzeren Messabstand zur Pflanze und eine kleinere Messfläche. Die Ergebnisse der beiden Sensicle-Spektrometer zeigten signifikant positive Korrelationen zwischen dem Ertrag und den Indizes von CropSpec, NDVI (Normalised Difference Vegetation Index) und REIP (Red-Edge Inflection Point). Auch FERARI und SFR (Simple Fluorescence Ratio) des MP-Sensors wurden für die Analyse des Ertragsvorhersagemodells ausgewählt. Mit dem verfügbaren N korrelierten CropSpec und REIP hochsignifikant. Das BM-Gewicht korrelierte bereits ab einem sehr frühen Wachstumsstadium (Z31) mit REIP und im Reifestadium (Z85) mit SAVI (Soil-Adjusted Vegetation Index). REIP, FERARI und SFR zeigten hohe Korrelationen mit dem verfügbaren N, insbesondere im Juni und Juli. Die Ratios und Signale des MP Sensors sind vor allem ab Z85 gegenüber dem BM-Gewicht hochsignifikant. Durch eine Kombination von Fluoreszenzwerten und Spektrometerindizes wurden lineare Modelle zur Vorhersage des Weizenertrags erstellt, die im Verlauf der Vegetationsperiode für das Versuchsfeld Lammwirt (LW) im Jahr 2012 signifikant korrelierten. Das beste Modell für das Feld LW im Jahr 2012 wurde für die Kreuzvalidierung mit den Messungen der Versuchsfelder Inneres Täle (IT) und Riech (RI) in den Jahren 2011 und 2012 ausgewählt. Sie waren nicht signifikant, jedoch zeigten sich durch den Austausch nur eines Spektralindexes mit einem Fluoreszenzratio in einem ähnlichen linearen Modell signifikante Korrelationen. Die vorliegende Arbeit zeigt erfolgreich, dass sich die Kombination verschiedener Sensorwerte und Sensorindizes zur Erkennung von Pflanzenmerkmalen gut eignet, und ohne den Einsatz destruktiver Methoden die Möglichkeit für bessere und robustere Vorhersagen bietet. Vor allem der MP-Fluoreszenzsensor erwies sich als universell einsetzbarer Sensor, der signifikante Korrelationen zu den untersuchten Merkmalen BM-Gewicht, Weizenertrag und verfügbarem N aufzeigte

Urban Drone Navigation: Autoencoder Learning Fusion for Aerodynamics

Drones are vital for urban emergency search and rescue (SAR) due to the challenges of navigating dynamic environments with obstacles like buildings and wind. This paper presents a method that combines multi-objective reinforcement learning (MORL) with a convolutional autoencoder to improve drone navigation in urban SAR. The approach uses MORL to achieve multiple goals and the autoencoder for cost-effective wind simulations. By utilizing imagery data of urban layouts, the drone can autonomously make navigation decisions, optimize paths, and counteract wind effects without traditional sensors. Tested on a New York City model, this method enhances drone SAR operations in complex urban settings.Comment: 47 page

Collaborative autonomy in heterogeneous multi-robot systems

As autonomous mobile robots become increasingly connected and widely deployed in different domains, managing multiple robots and their interaction is key to the future of ubiquitous autonomous systems. Indeed, robots are not individual entities anymore. Instead, many robots today are deployed as part of larger fleets or in teams. The benefits of multirobot collaboration, specially in heterogeneous groups, are multiple. Significantly higher degrees of situational awareness and understanding of their environment can be achieved when robots with different operational capabilities are deployed together. Examples of this include the Perseverance rover and the Ingenuity helicopter that NASA has deployed in Mars, or the highly heterogeneous robot teams that explored caves and other complex environments during the last DARPA Sub-T competition. This thesis delves into the wide topic of collaborative autonomy in multi-robot systems, encompassing some of the key elements required for achieving robust collaboration: solving collaborative decision-making problems; securing their operation, management and interaction; providing means for autonomous coordination in space and accurate global or relative state estimation; and achieving collaborative situational awareness through distributed perception and cooperative planning. The thesis covers novel formation control algorithms, and new ways to achieve accurate absolute or relative localization within multi-robot systems. It also explores the potential of distributed ledger technologies as an underlying framework to achieve collaborative decision-making in distributed robotic systems. Throughout the thesis, I introduce novel approaches to utilizing cryptographic elements and blockchain technology for securing the operation of autonomous robots, showing that sensor data and mission instructions can be validated in an end-to-end manner. I then shift the focus to localization and coordination, studying ultra-wideband (UWB) radios and their potential. I show how UWB-based ranging and localization can enable aerial robots to operate in GNSS-denied environments, with a study of the constraints and limitations. I also study the potential of UWB-based relative localization between aerial and ground robots for more accurate positioning in areas where GNSS signals degrade. In terms of coordination, I introduce two new algorithms for formation control that require zero to minimal communication, if enough degree of awareness of neighbor robots is available. These algorithms are validated in simulation and real-world experiments. The thesis concludes with the integration of a new approach to cooperative path planning algorithms and UWB-based relative localization for dense scene reconstruction using lidar and vision sensors in ground and aerial robots

Security Considerations in AI-Robotics: A Survey of Current Methods, Challenges, and Opportunities

Robotics and Artificial Intelligence (AI) have been inextricably intertwined since their inception. Today, AI-Robotics systems have become an integral part of our daily lives, from robotic vacuum cleaners to semi-autonomous cars. These systems are built upon three fundamental architectural elements: perception, navigation and planning, and control. However, while the integration of AI-Robotics systems has enhanced the quality our lives, it has also presented a serious problem - these systems are vulnerable to security attacks. The physical components, algorithms, and data that make up AI-Robotics systems can be exploited by malicious actors, potentially leading to dire consequences. Motivated by the need to address the security concerns in AI-Robotics systems, this paper presents a comprehensive survey and taxonomy across three dimensions: attack surfaces, ethical and legal concerns, and Human-Robot Interaction (HRI) security. Our goal is to provide users, developers and other stakeholders with a holistic understanding of these areas to enhance the overall AI-Robotics system security. We begin by surveying potential attack surfaces and provide mitigating defensive strategies. We then delve into ethical issues, such as dependency and psychological impact, as well as the legal concerns regarding accountability for these systems. Besides, emerging trends such as HRI are discussed, considering privacy, integrity, safety, trustworthiness, and explainability concerns. Finally, we present our vision for future research directions in this dynamic and promising field