Micro Aerial Vehicles (MAV) Assured Navigation in Search and Rescue Missions Robust Localization, Mapping and Detection

This Master's Thesis describes the developments on robust localization, mapping and detection algorithms for Micro Aerial Vehicles (MAVs). The localization method proposes a seamless indoor-outdoor multi-sensor architecture. This algorithm is capable of using all or a subset of its sensor inputs to determine a platform's position, velocity and attitude (PVA). It relies on the Inertial Measurement Unit as the core sensor and monitors the status and observability of the secondary sensors to select the most optimum estimator strategy for each situation. Furthermore, it ensures a smooth transition between filters structures. This document also describes the integration mechanism for a set of common sensors such as GNSS receivers, laser scanners and stereo and mono cameras. The mapping algorithm provides a fully automated fast aerial mapping pipeline. It speeds up the process by pre-selecting the images using the flight plan and the onboard localization. Furthermore, it relies on Structure from Motion (SfM) techniques to produce an optimized 3D reconstruction of camera locations and sparse scene geometry. These outputs are used to compute the perspective transformations that project the raw images on the ground and produce a geo-referenced map. Finally, these maps are fused with other domains in a collaborative UGV and UAV mapping algorithms. The real-time aerial detection of victims is based on a thermal camera. The algorithm is composed by three steps. Firstly, a normalization of the image is performed to get rid of the background and to extract the regions of interest. Later the victim detection and tracking steps produce the real-time geo-referenced locations of the detections. The thesis also proposes the concept of a MAV Copilot, a payload composed by a set of sensors and algorithm the enhances the capabilities of any commercial MAV. To develop and validate these contributions, a prototype of a search and rescue MAV and the Copilot has been developed. These developments have been validated in three large-scale demonstrations of search and rescue operations in the context of the European project ICARUS: a shipwreck in Lisbon (Portugal), an earthquake in Marche (Belgium), and the Fukushima nuclear disaster in the euRathlon 2015 competition in Piombino (Italy)

Reference Model for Interoperability of Autonomous Systems

This thesis proposes a reference model to describe the components of an Un-manned Air, Ground, Surface, or Underwater System (UxS), and the use of a single Interoperability Building Block to command, control, and get feedback from such vehicles. The importance and advantages of such a reference model, with a standard nomenclature and taxonomy, is shown. We overview the concepts of interoperability and some efforts to achieve common refer-ence models in other areas. We then present an overview of existing un-manned systems, their history, characteristics, classification, and missions. The concept of Interoperability Building Blocks (IBB) is introduced to describe standards, protocols, data models, and frameworks, and a large set of these are analyzed. A new and powerful reference model for UxS, named RAMP, is proposed, that describes the various components that a UxS may have. It is a hierarchical model with four levels, that describes the vehicle components, the datalink, and the ground segment. The reference model is validated by showing how it can be applied in various projects the author worked on. An example is given on how a single standard was capable of controlling a set of heterogeneous UAVs, USVs, and UGVs

Collaborative Multi-Robot Search and Rescue: Planning, Coordination, Perception, and Active Vision

Search and rescue (SAR) operations can take significant advantage from supporting autonomous or teleoperated robots and multi-robot systems. These can aid in mapping and situational assessment, monitoring and surveillance, establishing communication networks, or searching for victims. This paper provides a review of multi-robot systems supporting SAR operations, with system-level considerations and focusing on the algorithmic perspectives for multi-robot coordination and perception. This is, to the best of our knowledge, the first survey paper to cover (i) heterogeneous SAR robots in different environments, (ii) active perception in multi-robot systems, while (iii) giving two complementary points of view from the multi-agent perception and control perspectives. We also discuss the most significant open research questions: shared autonomy, sim-to-real transferability of existing methods, awareness of victims' conditions, coordination and interoperability in heterogeneous multi-robot systems, and active perception. The different topics in the survey are put in the context of the different challenges and constraints that various types of robots (ground, aerial, surface, or underwater) encounter in different SAR environments (maritime, urban, wilderness, or other post-disaster scenarios). The objective of this survey is to serve as an entry point to the various aspects of multi-robot SAR systems to researchers in both the machine learning and control fields by giving a global overview of the main approaches being taken in the SAR robotics area

Robotic equipment carrying RN detectors: requirements and capabilities for testing

77 pags., 32 figs., 5 tabs.-- ERNCIP Radiological and Nuclear Threats to Critical Infrastructure Thematic Group . -- This publication is a Technical report by the Joint Research Centre (JRC) . -- JRC128728 . -- EUR 31044 ENThe research leading to these results has received funding from the European Union as part of the European Reference Network for Critical Infrastructure Protection (ERNCIP) projec

Decentralized Collision-Free Control of Multiple Robots in 2D and 3D Spaces

Decentralized control of robots has attracted huge research interests. However, some of the research used unrealistic assumptions without collision avoidance. This report focuses on the collision-free control for multiple robots in both complete coverage and search tasks in 2D and 3D areas which are arbitrary unknown. All algorithms are decentralized as robots have limited abilities and they are mathematically proved. The report starts with the grid selection in the two tasks. Grid patterns simplify the representation of the area and robots only need to move straightly between neighbor vertices. For the 100% complete 2D coverage, the equilateral triangular grid is proposed. For the complete coverage ignoring the boundary effect, the grid with the fewest vertices is calculated in every situation for both 2D and 3D areas. The second part is for the complete coverage in 2D and 3D areas. A decentralized collision-free algorithm with the above selected grid is presented driving robots to sections which are furthest from the reference point. The area can be static or expanding, and the algorithm is simulated in MATLAB. Thirdly, three grid-based decentralized random algorithms with collision avoidance are provided to search targets in 2D or 3D areas. The number of targets can be known or unknown. In the first algorithm, robots choose vacant neighbors randomly with priorities on unvisited ones while the second one adds the repulsive force to disperse robots if they are close. In the third algorithm, if surrounded by visited vertices, the robot will use the breadth-first search algorithm to go to one of the nearest unvisited vertices via the grid. The second search algorithm is verified on Pioneer 3-DX robots. The general way to generate the formula to estimate the search time is demonstrated. Algorithms are compared with five other algorithms in MATLAB to show their effectiveness

A study to implement a 2D laser scanner to determine the platform position and orientation of a cable robot for logistic applications

Der Einsatz seilgetriebener Parallelmanipulatoren (CDPR) in der Industrie ist der Weg in eine vielversprechende Zukunft. Im Vergleich zum klassischen Parallelmanipulator hat der CDPR einen größeren Arbeitsraum und verbraucht dennoch weniger Energie. Ein CDPR-Prototyp für die Anwendung im Hochregallager wurde im Lehrstuhl für Mechatronik der Universität Duisburg-Essen entwickelt. Ziel ist, diese Anwendung auf den Markt zu bringen. Für die Roboterkalibrierung, zur Bewertung der Roboterregelung und aus Gründen der Sicherheit muss die Position und Orientierung (Pose) der CABLAR-Plattform bestimmt werden. Aktuelle Forschungsarbeiten zeigen, dass die effektivsten Verfahren zur Bestimmung der Plattformpose die direkte Messung mit einem externen Sensor und die indirekte Messung mit einem integrierten Sensor am Seilroboter sind. Beispiele für die direkte und indirekte Messung sind das Kamerasystem und die Vorwärtskinematik. Aufgrund der hohen Kosten ist das Kamerasystem für diese Anwendung weniger geeignet. Andererseits birgt auch die Vorwärtskinematik einige Nachteile: Die aktuelle Geometrie des Roboters stimmt wahrscheinlich wegen der Herstellungs- und/oder Montagetoleranz nicht mit dem kinematischen Modell überein. Zudem beeinflussen Umweltfaktoren, wie z. B. die Temperatur, und eine lange Betriebszeit die Seileigenschaften (z. B. Elastizitätsmodul, Seildichte, Durchmesser). Diese Änderungen verringern die Genauigkeit der Plattformpositionierung. Ein alternatives Verfahren zur Bestimmung der Plattformpose ist die Messung mittels eines 2D-Laserscanners und eines Orientierungsaufnehmers (IMU). In Kombination mit Reflektoren an der linken, rechten und spezielle Anordnung an oberen Seite des Roboterrahmens liefert der Laserscanner einzigartige Messdaten. Das Messergebnis des Laserscanners basiert auf dem Gerade-Ebene-Schnittpunkt, der mittels Gradientenprojektionsverfahren modelliert wird. Zudem wurde ein Kompensationsalgorithmus entwickelt, um die Auswirkung des Geschwindigkeitseffekts auf das Messergebnis aufgrund der Plattformbewegung zu verringern. Der Näherungswert der normalen Parametrierung aller Reflektoren wird mittels der modizierten Hough-Transformation geschätzt. Unter Zuhilfenahme dieses Wertes wurde das Messergebnis anhand eines zufälligen Stichprobenverfahrens (RANSAC Algorithmus, englisch Random Sample Consensus) segmentiert. Ziel ist, die Messdaten der Reflektoren an der linken, rechten, und oberen Seite des Roboterrahmens zu trennen. Die Methode der kleinsten Quadrate (KQ-Methode) bestimmt anhand dieser segmentierten Messdaten den besten Wert der normalen Parametrierung jeder Geraden, die zu allen Reflektoren gehört. Aus diesen Werten werden die y- und z-Komponente und der Rollwinkel der Plattformpose bestimmt. Um die Messfähigkeit des 2D-Laserscanners vom zwei dimensionalen zum räumlichen Messen zu erweitern, wurde ein mathematisches Modell mittels einer speziellen Reflektoranordnung entwickelt. Ziel ist die Bestimmung der x-Komponente und des Gierwinkels der Plattformpose. Der Nickwinkel wird vom IMU gemessen. Die Simulation der Plattform wurde in Ruhe und in Bewegung durchgeführt. Die Simulationsergebnisse sind als Empfehlung für den Versuch am Prototyp zu sehen. Vor dem Versuch wurde die passende Sensorschnittstelle gewählt und getestet. Zudem erfolgte die Gestaltung des Sensorkonzepts zur Datenübertragung. Der Treiber für die Sensoren und die Software für die Datenbearbeitung wurden vorbereitet und das vorgeschlagene Verfahren zur Bestimmung der Plattformpose am Prototyp getestet. Im Versuch wurde die translatorische Komponente der Plattformpose mit direkter Messung der Plattformposition validiert. Der Vergleich zeigt, dass die gemessene Plattformpose nicht an gewünschter Stelle liegt. Danach wurde das vorgeschlagene Verfahren zur Bestimmung der Plattformpose während niedriger und hoher Plattformgeschwindigkeit getestet. Das Ergebnis zeigt, dass dieses Verfahren zur Bestimmung der aktuellen Plattformpose geeignet ist. Die oben beschriebenen Forschungsergebnisse zeigen, dass vorgeschlagene Messverfahren sich zur Bestimmung der Plattformpose in Ruhe und in Bewegung eignet. Aufgrund des günstigen Preises ist das vorgeschlagene Messsystem eine vielsprechende Möglichkeit, die in der kommerziellen Anwendung des CABLAR eingesetzt werden kann.The implementation of a Cable Driven Parallel Robot (CDPR) as a commercial product has a promising future due to its energy efficiency and larger workspace compared to conventional parallel manipulators. A prototype of a CDPR for warehouse applications called CABLAR has been developed at the Chair of Mechatronics at the University of Duisburg-Essen (UDE), which aims to develop the CDPR as a commercial product. In order to benchmark the controller and calibrate the robot, for safety reasons, the platform position and orientation (pose) of the CABLAR must be measured. The current reported approaches to determine the platform pose of a cable robot are direct measurement and indirect measurement. An external sensor such as a camera system or laser tracker is used in direct measurement. Meanwhile, indirect measurement is the determination of the platform pose by forward kinematics where the input is from the proprioceptive sensor. However, the camera system is not worth implementing in the commercial product due to its high cost. On the other hand, forward kinematics has drawbacks when the defined parameters are not identical to the actual parameters. The manufacturing tolerance, assembly tolerance or changing the properties and diameter of the cable because of environmental effects (e.g. temperature) and long operation time are the reasons for this and are difficult to avoid. As a result, the actual pose of the platform could deviate from the desired pose. In this thesis, a direct measurement method by combining a 2D laser scanner with an Inertial Measurement Unit (IMU) is proposed. Several flat reflectors are fixed on the left and right of the robot frame with a special pattern design on the upper side. The laser scanner measurement result during the operation of the CABLAR is imitated based on the line-plane intersection according to the gradient projection method. A compensation algorithm aimed at reducing the velocity effect on the measurement result due to platform motion is proposed. The rough value of normal parametrization of each reflector is estimated using the Modified Hough Transform (mHT) from the measurement result. According to the rough value of the normal parametrization, the measurement result is segmented into a dataset corresponding to the left-, right- and upper-side reflectors by Random Sample Consensus (RANSAC). The linear least squares method is applied in order to determine the ne value of the normal parametrization of all segmented data. The y-component, z-component and roll angle of the platform pose are determined from the fine normal parametrization. A mathematical model based on the reflector special pattern design is developed. The goal is to extend the limitation of the 2D laser scanner from plane measurement to become space measurement in order to obtain the x-component and the yaw angle of the platform pose. The pitch angle is measured by the IMU. The CABLAR model is simulated to verify the proposed measurement method for the conditions where the platform is stationary and in motion. According to the simulation results, several points are concluded as the recommendations for the experiment. Before the experiment was conducted, the suitable hardware interface of the sensors was chosen and tested. The system architecture of the data transfer was designed. The software to drive the sensors and to process the measurement data was prepared. In the experiment on the prototype, the translational components of the platform pose were validated with the direct measurement. Meanwhile the rotational components obtained from the proposed method were validated with the measurement result from the IMU. The results show that the platform position deviates from the desired pose. Furthermore, the proposed platform pose measurement method is tested for the platform in low- and high-velocity motion. The results show that the proposed measurement method is able to determine the actual platform pose. Finally, the proposed measurement method is able to determine the platform pose when stationary and in motion. The proposed measurement system is suitable for application in the commercial CABLAR due its low cost compared to the actual reported measurement system

Autonomous reconnaissance and surveillance in urban structures - Eurathlon 2013

In this paper we propose an integrated hard- and software system for autonomous exploration and mapping of dilapidated buildings. The system is based on well understood approaches towards SLAM and exploration. Because of the real world nature of the application, a fragile wireless connection and difficult obstacles like staircases were taken into account during system design. Additionally, a drop-off system for Wi-Fi relays was introduced in order to increase the communication range. Further, a homing function is added to safely explore beyond radio coverage. Negative obstacles are handled using a tilted laserscanner. A semi-automatic stair climbing mechanism allows to travel between floors. The system is interconnected with a control station, giving an operator a fine grained control over the mission without disturbing the exploration and mapping task. The system was successfully tested at the Eurathlon 2013 robotics competition under real world conditions