Formation Navigation and Relative Localisation of Multi-Robot Systems

When proceeding from single to multiple robots, cooperative action is one of the most relevant topics. The domain of robotic security systems contains typical applications for a multi-robot system (MRS). Possible scenarios are safety and security issues on airports, harbours, large industry plants or museums. Additionally, the field of environmental supervision is an up-coming issue. Inherent to these applications is the need for an organised and coordinated navigation of the robots, and a vital prerequisite for any coordinated movements is a good localisation. This dissertation will present novel approaches to the problems of formation navigation and relative localisation with multiple ground-based mobile robots. It also looks into the question what kind of metric is applicable for multi-robot navigation problems. Thereby, the focus of this work will be on aspects of 1. coordinated navigation and movement A new potential-field-based approach to formation navigation is presented. In contradiction to classical potential-field-based formation approaches, the proposed method also uses the orientation between neighbours in the formation. Consequently, each robot has a designated position within the formation. Therefore, the new method is called directed potential field approach. Extensive experiments prove that the method is capable of generating all kinds of formation shapes, even in the presence of dense obstacles. All tests have been conducted with simulated and real robots and successfully guided the robot formation through environments with varying obstacle configurations. In comparison, the nondirected potential field approach turns out to be unstable regarding the positions of the robots within formations. The robots strive to switch their positions, e.g. when passing through narrow passages. Under such conditions the directed approach shows a preferable behaviour, called “breathing”. The formation shrinks or inflates depending on the obstacle situation while trying to maintain its shape and keep the robots at their desired positions inside the formation. For a more particular comparison of formation algorithms it is important to have measures that allow a meaningful evaluation of the experimental data. For this purpose a new formation metric is developed. If there are many obstacles, the formation error must be scaled down to be comparable to an empty environment where the error would be small. Assuming that the environment is unknown and possibly non-static, only actual sensor information can be used for these calculations. We developed a special weighting factor, which is inverse proportional to the “density” of obstacles and which turns out to model the influence of the environment adequately. 2. relative localisation A new method for relative localisation between the members of a robot group is introduced. This relative localisation approach uses mutual sensor observations to localise the robots with respect to other objects – without having an environment model. Techniques like the Extended Kalman Filter (EKF) have proven to be powerful tools in the field of single robot applications. This work presents extensions to these algorithms with respect to the use in MRS. These aspects are investigated and combined under the topic of improving and stabilising the performance of the localisation and navigation process. Most of the common localisation approaches use maps and/or landmarks with the intention of generating a globally consistent world-coordinate system for the robot group. The aim of the here presented relative localisation approach, on the other hand, is to maintain only relative positioning between the robots. The presented method enables a group of mobile robots to start at an unknown location in an unknown environment and then to incrementally estimate their own positions and the relative locations of the other robots using only sensor information. The result is a robust, fast and precise approach, which does not need any preconditions or special assumptions about the environment. To validate the approach extensive tests with both, real and simulated, robots have been conducted. For a more specific evaluation, the Mean Localisation Error (MLE) is introduced. The conducted experiments include a comparison between the proposed Extended Kalman Filter and a standard SLAM-based approach. The developed method robustly delivered an accuracy better than 2 cm and performed at least as well as the SLAM approach. The algorithm coped with scattered groups of robots while moving on arbitrarily shaped paths. In summary, this thesis presents novel approaches to the field of coordinated navigation in multi-robot systems. The results facilitate cooperative movements of robot groups as well as relative localisation among the group members. In addition, a solid foundation for a non-environment related metric for formation navigation is introduced

Cooperative monocular-based SLAM for multi-UAV systems in GPS-denied environments

This work presents a cooperative monocular-based SLAM approach for multi-UAV systems that can operate in GPS-denied environments. The main contribution of the work is to show that, using visual information obtained from monocular cameras mounted onboard aerial vehicles flying in formation, the observability properties of the whole system are improved. This fact is especially notorious when compared with other related visual SLAM configurations. In order to improve the observability properties, some measurements of the relative distance between the UAVs are included in the system. These relative distances are also obtained from visual information. The proposed approach is theoretically validated by means of a nonlinear observability analysis. Furthermore, an extensive set of computer simulations is presented in order to validate the proposed approach. The numerical simulation results show that the proposed system is able to provide a good position and orientation estimation of the aerial vehicles flying in formation.Peer ReviewedPostprint (published version

Probablistic approaches for intelligent AUV localisation

This thesis studies the problem of intelligent localisation for an autonomous underwater vehicle (AUV). After an introduction about robot localisation and specific issues in the underwater domain, the thesis will focus on passive techniques for AUV localisation, highlighting experimental results and comparison among different techniques. Then, it will develop active techniques, which require intelligent decisions about the steps to undertake in order for the AUV to localise itself. The undertaken methodology consisted in three stages: theoretical analysis of the problem, tests with a simulation environment, integration in the robot architecture and field trials. The conclusions highlight applications and scenarios where the developed techniques have been successfully used or can be potentially used to enhance the results given by current techniques. The main contribution of this thesis is in the proposal of an active localisation module, which is able to determine the best set of action to be executed, in order to maximise the localisation results, in terms of time and efficiency

Communication-based UAV Swarm Missions

Unmanned aerial vehicles have developed rapidly in recent years due to technological advances. UAV technology can be applied to a wide range of applications in surveillance, rescue, agriculture and transport. The problems that can exist in these areas can be mitigated by combining clusters of drones with several technologies. For example, when a swarm of drones is under attack, it may not be able to obtain the position feedback provided by the Global Positioning System (GPS). This poses a new challenge for the UAV swarm to fulfill a specific mission. This thesis intends to use as few sensors as possible on the UAVs and to design the smallest possible information transfer between the UAVs to maintain the shape of the UAV formation in flight and to follow a predetermined trajectory. This thesis presents Extended Kalman Filter methods to navigate autonomously in a GPS-denied environment. The UAV formation control and distributed communication methods are also discussed and given in detail

Visual localisation of electricity pylons for power line inspection

Inspection of power infrastructure is a regular maintenance event. To date the inspection process has mostly been done manually, but there is growing interest in automating the process. The automation of the inspection process will require an accurate means for the localisation of the power infrastructure components. In this research, we studied the visual localisation of a pylon. The pylon is the most prominent component of the power infrastructure and can provide a context for the inspection of the other components. Point-based descriptors tend to perform poorly on texture less objects such as pylons, therefore we explored the localisation using convolutional neural networks and geometric constraints. The crossings of the pylon, or vertices, are salient points on the pylon. These vertices aid with recognition and pose estimation of the pylon. We were successfully able to use a convolutional neural network for the detection of the vertices. A model-based technique, geometric hashing, was used to establish the correspondence between the stored pylon model and the scene object. We showed the effectiveness of the method as a voting technique to determine the pose estimation from a single image. In a localisation framework, the method serves as the initialization of the tracking process. We were able to incorporate an extended Kalman filter for subsequent incremental tracking of the camera relative to the pylon. Also, we demonstrated an alternative tracking using heatmap details from the vertex detection. We successfully demonstrated the proposed algorithms and evaluated their effectiveness using a model pylon we built in the laboratory. Furthermore, we revalidated the results on a real-world outdoor electricity pylon. Our experiments illustrate that model-based techniques can be deployed as part of the navigation aspect of a robot

Full State History Cooperative Localisation with Complete Information Sharing

This thesis presents a decentralised localisation method for multiple robots. We enable reduced bandwidth requirements whilst using local solutions that fuse information from other robots. This method does not specify a communication topology or require complex tracking of information. The methods for including shared data match standard elements of nonlinear optimisation algorithms. There are four contributions in this thesis. The first is a method to split the multiple vehicle problem into sections that can be iteratively transmitted in packets with bandwidth bounds. This is done through delayed elimination of external states, which are states involved in intervehicle observations. Observations are placed in subgraphs that accumulate between external states. Internal states, which are all states not involved in intervehicle observations, can then be eliminated from each subgraph and the joint probability of the start and end states is shared between vehicles and combined to yield the solution to the entire graph. The second contribution is usage of variable reordering within these packets to enable handling of delayed observations that target an existing state such as with visual loop closures. We identify the calculations required to give the conditional probability of the delayed historical state on the existing external states before and after. This reduces the recalculation to updating the factorisation of a single subgraph and is independent of the time since the observation was made. The third contribution is a method and conditions for insertion of states into existing packets that does not invalidate previously transmitted data. We derive the conditions that enable this method and our fourth contribution is two motion models that conform to the conditions. Together this permits handling of the general out of sequence case

Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion

Agricultural mobile robots have great potential to effectively implement different agricultural tasks. They can save human labour costs, avoid the need for people having to perform risky operations and increase productivity. Automation and advanced sensing technologies can provide up-to-date information that helps farmers in orchard management. Data collected from on-board sensors on a mobile robot provide information that can help the farmer detect tree or fruit diseases or damage, measure tree canopy volume and monitor fruit development. In orchards, trees are natural landmarks providing suitable cues for mobile robot localisation and navigation as trees are nominally planted in straight and parallel rows. This thesis presents a novel tree trunk detection algorithm that detects trees and discriminates between trees and non-tree objects in the orchard using a camera and 2D laser scanner data fusion. A local orchard map of the individual trees was developed allowing the mobile robot to navigate to a specific tree in the orchard to perform a specific task such as tree inspection. Furthermore, this thesis presents a localisation algorithm that does not rely on GPS positions and depends only on the on-board sensors of the mobile robot without adding any artificial landmarks, respective tapes or tags to the trees. The novel tree trunk detection algorithm combined the features extracted from a low cost camera's images and 2D laser scanner data to increase the robustness of the detection. The developed algorithm used a new method to detect the edge points and determine the width of the tree trunks and non-tree objects from the laser scan data. Then a projection of the edge points from the laser scanner coordinates to the image plane was implemented to construct a region of interest with the required features for tree trunk colour and edge detection. The camera images were used to verify the colour and the parallel edges of the tree trunks and non-tree objects. The algorithm automatically adjusted the colour detection parameters after each test which was shown to increase the detection accuracy. The orchard map was constructed based on tree trunk detection and consisted of the 2D positions of the individual trees and non-tree objects. The map of the individual trees was used as an a priority map for mobile robot localisation. A data fusion algorithm based on an Extended Kalman filter was used for pose estimation of the mobile robot in different paths (midway between rows, close to the rows and moving around trees in the row) and different turns (semi-circle and right angle turns) required for tree inspection tasks. The 2D positions of the individual trees were used in the correction step of the Extended Kalman filter to enhance localisation accuracy. Experimental tests were conducted in a simulated environment and a real orchard to evaluate the performance of the developed algorithms. The tree trunk detection algorithm was evaluated under two broad illumination conditions (sunny and cloudy). The algorithm was able to detect the tree trunks (regular and thin tree trunks) and discriminate between trees and non-tree objects with a detection accuracy of 97% showing that the fusion of both vision and 2D laser scanner technologies produced robust tree trunk detection. The mapping method successfully localised all the trees and non-tree objects of the tested tree rows in the orchard environment. The mapping results indicated that the constructed map can be reliably used for mobile robot localisation and navigation. The localisation algorithm was evaluated against the logged RTK-GPS positions for different paths and headland turns. The average of the RMS of the position error in x, y coordinates and Euclidean distance were 0.08 m, 0.07 m and 0.103 m respectively, whilst the average of the RMS of the heading error was 3:32°. These results were considered acceptable while driving along the rows and when executing headland turns for the target application of autonomous mobile robot navigation and tree inspection tasks in orchards