Collaborative autonomy in heterogeneous multi-robot systems

As autonomous mobile robots become increasingly connected and widely deployed in different domains, managing multiple robots and their interaction is key to the future of ubiquitous autonomous systems. Indeed, robots are not individual entities anymore. Instead, many robots today are deployed as part of larger fleets or in teams. The benefits of multirobot collaboration, specially in heterogeneous groups, are multiple. Significantly higher degrees of situational awareness and understanding of their environment can be achieved when robots with different operational capabilities are deployed together. Examples of this include the Perseverance rover and the Ingenuity helicopter that NASA has deployed in Mars, or the highly heterogeneous robot teams that explored caves and other complex environments during the last DARPA Sub-T competition. This thesis delves into the wide topic of collaborative autonomy in multi-robot systems, encompassing some of the key elements required for achieving robust collaboration: solving collaborative decision-making problems; securing their operation, management and interaction; providing means for autonomous coordination in space and accurate global or relative state estimation; and achieving collaborative situational awareness through distributed perception and cooperative planning. The thesis covers novel formation control algorithms, and new ways to achieve accurate absolute or relative localization within multi-robot systems. It also explores the potential of distributed ledger technologies as an underlying framework to achieve collaborative decision-making in distributed robotic systems. Throughout the thesis, I introduce novel approaches to utilizing cryptographic elements and blockchain technology for securing the operation of autonomous robots, showing that sensor data and mission instructions can be validated in an end-to-end manner. I then shift the focus to localization and coordination, studying ultra-wideband (UWB) radios and their potential. I show how UWB-based ranging and localization can enable aerial robots to operate in GNSS-denied environments, with a study of the constraints and limitations. I also study the potential of UWB-based relative localization between aerial and ground robots for more accurate positioning in areas where GNSS signals degrade. In terms of coordination, I introduce two new algorithms for formation control that require zero to minimal communication, if enough degree of awareness of neighbor robots is available. These algorithms are validated in simulation and real-world experiments. The thesis concludes with the integration of a new approach to cooperative path planning algorithms and UWB-based relative localization for dense scene reconstruction using lidar and vision sensors in ground and aerial robots

Behaviour monitoring: investigation of local and distributed approaches

Nowadays, the widespread availability of cheap and efficient unmanned systems (either aerial, ground or surface) has led to significant opportunities in the field of remote sensing and automated monitoring. On the one hand, the definition of efficient approaches to information collection, filtering and fusion has been the focus of extremely relevant research streams over the last decades. On the other hand, far less attention has been given to the problem of ‘interpreting’ the data, thus implementing inference processes able to, e.g., spot anomalies and possible threats in the monitored scenario. It is easy to understand how the automation of the ‘target assessment’ process could bring a great impact on monitoring applications since it would allow sensibly alleviating the analysis burden for human operators. To this end, the research project proposed in this thesis addresses the problem of behaviour assessment leading to the identification of targets that exhibit features “of interest”. Firstly, this thesis has addressed the problem of distributed target assessment based on behavioural and contextual features. The assessment problem is analysed making reference to a layered structure and a possible implementation approach for the middle-layer has been proposed. An extensive analysis of the ‘feature’ concept is provided, together with considerations about the target assessment process. A case study considering a road-traffic monitoring application is then introduced, suggesting a possible implementation for a set of features related to this particular scenario. The distributed approach has been implemented employing a consensus protocol, which allows achieving agreement about high-level, non-measurable, characteristics of the monitored vehicles. Two different techniques, ‘Belief’ and ‘Average’ consensus, for distributed target assessment based on features are finally presented, enabling the comparison of consensus effects when implemented at different level of the considered conceptual hierarchy. Then, the problem of identifying targets concerning features is tackled using a different approach: a probabilistic description is adopted for the target characteristics of interest and a hypothesis testing technique is applied to the feature probability density functions. Such approach is expected to allow discerning whether a given vehicle is a target of interest or not. The assessment process introduced is also able to account for information about the context of the vehicle, i.e. the environment where it moves or is operated. In so doing the target assessment process can be effectively adapted to the contour conditions. Results from simulations involving a road monitoring scenario are presented, considering both synthetic and real-world data. Lastly, the thesis addresses the problem of manoeuvre recognition and behaviour anomalies detection for generic targets through pattern matching techniques. This problem is analysed considering motor vehicles in a multi-lane road scenario. The proposed approach, however, can be easily extended to significantly different monitoring contexts. The overall proposed solution consists in a trajectory analysis tool, which classifies the target position over time into a sequence of ‘driving modes’, and a string-matching technique. This classification allows, as result of two different approaches, detecting both a priori defined patterns of interest and general behaviours standing out from those regularly exhibited from the monitored targets. Regarding the pattern matching process, two techniques are introduced and compared: a basic approach based on simple strings and a newly proposed method based on ‘regular expressions’. About reference patterns, a technique for the automatic definition of a dictionary of regular expressions matching the commonly observed target manoeuvres is presented. Its assessment results are then compared to those of a classic multi-layered neural network. In conclusion, this thesis proposes some novel approaches, both local and distributed, for the identification of the ‘targets of interest’ within a multi-target scenario. Such assessment is solely based on the behaviour actually exhibited by a target and does not involve any specific knowledge about the targets (analytic dynamic models, previous data, signatures of any type, etc.), being thus easily applicable to different scenarios and target types. For all the novel approaches described in the thesis, numerical results from simulations are reported: these results, in all the cases, confirm the effectiveness of the proposed techniques, even if they appear to be open to interpretation because of the inherent subjectivity of the assessment process

Indoor Positioning and Navigation

In recent years, rapid development in robotics, mobile, and communication technologies has encouraged many studies in the field of localization and navigation in indoor environments. An accurate localization system that can operate in an indoor environment has considerable practical value, because it can be built into autonomous mobile systems or a personal navigation system on a smartphone for guiding people through airports, shopping malls, museums and other public institutions, etc. Such a system would be particularly useful for blind people. Modern smartphones are equipped with numerous sensors (such as inertial sensors, cameras, and barometers) and communication modules (such as WiFi, Bluetooth, NFC, LTE/5G, and UWB capabilities), which enable the implementation of various localization algorithms, namely, visual localization, inertial navigation system, and radio localization. For the mapping of indoor environments and localization of autonomous mobile sysems, LIDAR sensors are also frequently used in addition to smartphone sensors. Visual localization and inertial navigation systems are sensitive to external disturbances; therefore, sensor fusion approaches can be used for the implementation of robust localization algorithms. These have to be optimized in order to be computationally efficient, which is essential for real-time processing and low energy consumption on a smartphone or robot

Model-Augmented Haptic Telemanipulation: Concept, Retrospective Overview, and Current Use Cases

Certain telerobotic applications, including telerobotics in space, pose particularly demanding challenges to both technology and humans. Traditional bilateral telemanipulation approaches often cannot be used in such applications due to technical and physical limitations such as long and varying delays, packet loss, and limited bandwidth, as well as high reliability, precision, and task duration requirements. In order to close this gap, we research model-augmented haptic telemanipulation (MATM) that uses two kinds of models: a remote model that enables shared autonomous functionality of the teleoperated robot, and a local model that aims to generate assistive augmented haptic feedback for the human operator. Several technological methods that form the backbone of the MATM approach have already been successfully demonstrated in accomplished telerobotic space missions. On this basis, we have applied our approach in more recent research to applications in the fields of orbital robotics, telesurgery, caregiving, and telenavigation. In the course of this work, we have advanced specific aspects of the approach that were of particular importance for each respective application, especially shared autonomy, and haptic augmentation. This overview paper discusses the MATM approach in detail, presents the latest research results of the various technologies encompassed within this approach, provides a retrospective of DLR's telerobotic space missions, demonstrates the broad application potential of MATM based on the aforementioned use cases, and outlines lessons learned and open challenges

Applications

Volume 3 describes how resource-aware machine learning methods and techniques are used to successfully solve real-world problems. The book provides numerous specific application examples: in health and medicine for risk modelling, diagnosis, and treatment selection for diseases in electronics, steel production and milling for quality control during manufacturing processes in traffic, logistics for smart cities and for mobile communications

Applications

Volume 3 describes how resource-aware machine learning methods and techniques are used to successfully solve real-world problems. The book provides numerous specific application examples: in health and medicine for risk modelling, diagnosis, and treatment selection for diseases in electronics, steel production and milling for quality control during manufacturing processes in traffic, logistics for smart cities and for mobile communications