Adaptive Localization and Mapping for Planetary Rovers

Future rovers will be equipped with substantial onboard autonomy as space agencies and industry proceed with missions studies and technology development in preparation for the next planetary exploration missions. Simultaneous Localization and Mapping (SLAM) is a fundamental part of autonomous capabilities and has close connections to robot perception, planning and control. SLAM positively affects rover operations and mission success. The SLAM community has made great progress in the last decade by enabling real world solutions in terrestrial applications and is nowadays addressing important challenges in robust performance, scalability, high-level understanding, resources awareness and domain adaptation. In this thesis, an adaptive SLAM system is proposed in order to improve rover navigation performance and demand. This research presents a novel localization and mapping solution following a bottom-up approach. It starts with an Attitude and Heading Reference System (AHRS), continues with a 3D odometry dead reckoning solution and builds up to a full graph optimization scheme which uses visual odometry and takes into account rover traction performance, bringing scalability to modern SLAM solutions. A design procedure is presented in order to incorporate inertial sensors into the AHRS. The procedure follows three steps: error characterization, model derivation and filter design. A complete kinematics model of the rover locomotion subsystem is developed in order to improve the wheel odometry solution. Consequently, the parametric model predicts delta poses by solving a system of equations with weighed least squares. In addition, an odometry error model is learned using Gaussian processes (GPs) in order to predict non-systematic errors induced by poor traction of the rover with the terrain. The odometry error model complements the parametric solution by adding an estimation of the error. The gained information serves to adapt the localization and mapping solution to the current navigation demands (domain adaptation). The adaptivity strategy is designed to adjust the visual odometry computational load (active perception) and to influence the optimization back-end by including highly informative keyframes in the graph (adaptive information gain). Following this strategy, the solution is adapted to the navigation demands, providing an adaptive SLAM system driven by the navigation performance and conditions of the interaction with the terrain. The proposed methodology is experimentally verified on a representative planetary rover under realistic field test scenarios. This thesis introduces a modern SLAM system which adapts the estimated pose and map to the predicted error. The system maintains accuracy with fewer nodes, taking the best of both wheel and visual methods in a consistent graph-based smoothing approach

RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control

We present a unified model-based and data-driven approach for quadrupedal planning and control to achieve dynamic locomotion over uneven terrain. We utilize on-board proprioceptive and exteroceptive feedback to map sensory information and desired base velocity commands into footstep plans using a reinforcement learning (RL) policy trained in simulation over a wide range of procedurally generated terrains. When ran online, the system tracks the generated footstep plans using a model-based controller. We evaluate the robustness of our method over a wide variety of complex terrains. It exhibits behaviors which prioritize stability over aggressive locomotion. Additionally, we introduce two ancillary RL policies for corrective whole-body motion tracking and recovery control. These policies account for changes in physical parameters and external perturbations. We train and evaluate our framework on a complex quadrupedal system, ANYmal version B, and demonstrate transferability to a larger and heavier robot, ANYmal C, without requiring retraining.Comment: 19 pages, 15 figures, 6 tables, 1 algorithm, submitted to T-RO; under revie

Location-Enabled IoT (LE-IoT): A Survey of Positioning Techniques, Error Sources, and Mitigation

The Internet of Things (IoT) has started to empower the future of many industrial and mass-market applications. Localization techniques are becoming key to add location context to IoT data without human perception and intervention. Meanwhile, the newly-emerged Low-Power Wide-Area Network (LPWAN) technologies have advantages such as long-range, low power consumption, low cost, massive connections, and the capability for communication in both indoor and outdoor areas. These features make LPWAN signals strong candidates for mass-market localization applications. However, there are various error sources that have limited localization performance by using such IoT signals. This paper reviews the IoT localization system through the following sequence: IoT localization system review -- localization data sources -- localization algorithms -- localization error sources and mitigation -- localization performance evaluation. Compared to the related surveys, this paper has a more comprehensive and state-of-the-art review on IoT localization methods, an original review on IoT localization error sources and mitigation, an original review on IoT localization performance evaluation, and a more comprehensive review of IoT localization applications, opportunities, and challenges. Thus, this survey provides comprehensive guidance for peers who are interested in enabling localization ability in the existing IoT systems, using IoT systems for localization, or integrating IoT signals with the existing localization sensors

Visual attention and swarm cognition for off-road robots

Tese de doutoramento, Informática (Engenharia Informática), Universidade de Lisboa, Faculdade de Ciências, 2011Esta tese aborda o problema da modelação de atenção visual no contexto de robôs autónomos todo-o-terreno. O objectivo de utilizar mecanismos de atenção visual é o de focar a percepção nos aspectos do ambiente mais relevantes à tarefa do robô. Esta tese mostra que, na detecção de obstáculos e de trilhos, esta capacidade promove robustez e parcimónia computacional. Estas são características chave para a rapidez e eficiência dos robôs todo-o-terreno. Um dos maiores desafios na modelação de atenção visual advém da necessidade de gerir o compromisso velocidade-precisão na presença de variações de contexto ou de tarefa. Esta tese mostra que este compromisso é resolvido se o processo de atenção visual for modelado como um processo auto-organizado, cuja operação é modulada pelo módulo de selecção de acção, responsável pelo controlo do robô. Ao fechar a malha entre o processo de selecção de acção e o de percepção, o último é capaz de operar apenas onde é necessário, antecipando as acções do robô. Para fornecer atenção visual com propriedades auto-organizadas, este trabalho obtém inspiração da Natureza. Concretamente, os mecanismos responsáveis pela capacidade que as formigas guerreiras têm de procurar alimento de forma auto-organizada, são usados como metáfora na resolução da tarefa de procurar, também de forma auto-organizada, obstáculos e trilhos no campo visual do robô. A solução proposta nesta tese é a de colocar vários focos de atenção encoberta a operar como um enxame, através de interacções baseadas em feromona. Este trabalho representa a primeira realização corporizada de cognição de enxame. Este é um novo campo de investigação que procura descobrir os princípios básicos da cognição, inspeccionando as propriedades auto-organizadas da inteligência colectiva exibida pelos insectos sociais. Logo, esta tese contribui para a robótica como disciplina de engenharia e para a robótica como disciplina de modelação, capaz de suportar o estudo do comportamento adaptável.Esta tese aborda o problema da modelação de atenção visual no contexto de robôs autónomos todo-o-terreno. O objectivo de utilizar mecanismos de atenção visual é o de focar a percepção nos aspectos do ambiente mais relevantes à tarefa do robô. Esta tese mostra que, na detecção de obstáculos e de trilhos, esta capacidade promove robustez e parcimónia computacional. Estas são características chave para a rapidez e eficiência dos robôs todo-o-terreno. Um dos maiores desafios na modelação de atenção visual advém da necessidade de gerir o compromisso velocidade-precisão na presença de variações de contexto ou de tarefa. Esta tese mostra que este compromisso é resolvido se o processo de atenção visual for modelado como um processo auto-organizado, cuja operação é modulada pelo módulo de selecção de acção, responsável pelo controlo do robô. Ao fechar a malha entre o processo de selecção de acção e o de percepção, o último é capaz de operar apenas onde é necessário, antecipando as acções do robô. Para fornecer atenção visual com propriedades auto-organizadas, este trabalho obtém inspi- ração da Natureza. Concretamente, os mecanismos responsáveis pela capacidade que as formi- gas guerreiras têm de procurar alimento de forma auto-organizada, são usados como metáfora na resolução da tarefa de procurar, também de forma auto-organizada, obstáculos e trilhos no campo visual do robô. A solução proposta nesta tese é a de colocar vários focos de atenção encoberta a operar como um enxame, através de interacções baseadas em feromona. Este trabalho representa a primeira realização corporizada de cognição de enxame. Este é um novo campo de investigação que procura descobrir os princípios básicos da cognição, ins- peccionando as propriedades auto-organizadas da inteligência colectiva exibida pelos insectos sociais. Logo, esta tese contribui para a robótica como disciplina de engenharia e para a robótica como disciplina de modelação, capaz de suportar o estudo do comportamento adaptável.Fundação para a Ciência e a Tecnologia (FCT,SFRH/BD/27305/2006); Laboratory of Agent Modelling (LabMag

Shallow neural networks for autonomous robots

The use of Neural Networks (NNs) in modern applications is already well established thanks to the technological advancements in processing units and Deep Learning (DL), as well as the availability of deployment frameworks and services. However, the embedding of these methods in robotic systems is problematic when it comes to field operations. The use of Graphics Processing Units (GPUs) for such networks requires high amounts of power which would lead to shortened operational times. This is not desired since autonomous robots already need to manage their power supply to accommodate the lengths of their missions which can extend from hours to days. While external processing is possible, real-time monitoring can become unfeasible where delays are present. This also applies to autonomous robots that are deployed for underwater or space missions. For these reasons, there is a requirement for shallow but robust NN-based solutions that enhance the autonomy of a robot. This dissertation focuses on the design and meticulous parametrization complemented by methods that explain hyper-parameter importance. This is performed in the context of different settings and problems for autonomous robots in field operations. The contribution of this thesis comes in the form of autonomy augmentation for robots through shallow NNs that can potentially be embedded in future systems carrying NN processing units. This is done by implementing neural architectures that use sensor data to extract representations for event identification and learn patterns for event anticipation. This work harnesses Long Short-Term Memory networks (LSTMs) as the underpinning framework for time series representation and interpretation. This has been tested in three significant problems found in field operations: hardware malfunction classification, survey trajectory classification and hazardous event forecast and detection

GPGM-SLAM: a Robust SLAM System for Unstructured Planetary Environments with Gaussian Process Gradient Maps

Simultaneous Localization and Mapping (SLAM) techniques play a key role towards long-term autonomy of mobile robots due to the ability to correct localization errors and produce consistent maps of an environment over time. Contrarily to urban or man-made environments, where the presence of unique objects and structures offer unique cues for localization, the apperance of unstructured natural environments is often ambiguous and self-similar, hindering the performances of loop closure detection. In this paper, we present an approach to improve the robustness of place recognition in the context of a submap-based stereo SLAM based on Gaussian Process Gradient Maps (GPGMaps). GPGMaps embed a continuous representation of the gradients of the local terrain elevation by means of Gaussian Process regression and Structured Kernel Interpolation, given solely noisy elevation measurements. We leverage the imagelike structure of GPGMaps to detect loop closures using traditional visual features and Bag of Words. GPGMap matching is performed as an SE(2) alignment to establish loop closure constraints within a pose graph. We evaluate the proposed pipeline on a variety of datasets recorded on Mt. Etna, Sicily and in the Morocco desert, respectively Moon- and Mars-like environments, and we compare the localization performances with state-of-the-art approaches for visual SLAM and visual loop closure detection

MODEL PREDICTIVE CONTROL OF SKID-STEERED MOBILE ROBOT WITH DEEP LEARNING SYSTEM DYNAMICS

This thesis project presents several model predictive control (MPC) strategies for control of skid-steered mobile robots (SSMRs) using two different combinations of software environment, optimization tool and machine learning framework. The control strategies are tested in WeBots simulator. Spatial-based path following MPC of SSMR with static obstacle avoidance is developed in MATLAB environment with ACADO optimization toolkit using spatial kinematic model of SSMR. It includes static obstacle and border avoidance strategy based on artificial potential fields. Simulations show that the controller is effective at driving SSMR on a track, while avoiding borders and obstacles. Several more MPCs are developed using Python environment, ACADOS optimisation framework, and Pytorch-Casadi integration framework. Two time-domain controllers are made in Python environment, one based on SSMR kinematic model and another based on data-driven state-space model using Pytorch- Casadi framework. Both are setup to reach a goal point in simulation experiment. Experiments show that both versions reliably reach a target point. Standard and data-driven versions of spatial path following MPC are developed. Standard is a reimplementation of MPC designed in MATLAB with modifications to cost function and border avoidance, without static obstacle avoidance. Data-driven path following MPC is an extension of standard variant with state-space model replaced with a hybrid of spatial kinematics and data-driven model. Simulation of both spatial controllers confirm their effectiveness in following reference path

Planetary Rover Inertial Navigation Applications: Pseudo Measurements and Wheel Terrain Interactions

Accurate localization is a critical component of any robotic system. During planetary missions, these systems are often limited by energy sources and slow spacecraft computers. Using proprioceptive localization (e.g., using an inertial measurement unit and wheel encoders) without external aiding is insufficient for accurate localization. This is mainly due to the integrated and unbounded errors of the inertial navigation solutions and the drifted position information from wheel encoders caused by wheel slippage. For this reason, planetary rovers often utilize exteroceptive (e.g., vision-based) sensors. On the one hand, localization with proprioceptive sensors is straightforward, computationally efficient, and continuous. On the other hand, using exteroceptive sensors for localization slows rover driving speed, reduces rover traversal rate, and these sensors are sensitive to the terrain features. Given the advantages and disadvantages of both methods, this thesis focuses on two objectives. First, improving the proprioceptive localization performance without significant changes to the rover operations. Second, enabling adaptive traversability rate based on the wheel-terrain interactions while keeping the localization reliable. To achieve the first objective, we utilized the zero-velocity, zero-angular rate updates, and non-holonomicity of a rover to improve rover localization performance even with the limited available sensor usage in a computationally efficient way. Pseudo-measurements generated from proprioceptive sensors when the rover is stationary conditions and the non-holonomic constraints while traversing can be utilized to improve the localization performance without any significant changes to the rover operations. Through this work, it is observed that a substantial improvement in localization performance, without the aid of additional exteroceptive sensor information. To achieve the second objective, the relationship between the estimation of localization uncertainty and wheel-terrain interactions through slip-ratio was investigated. This relationship was exposed with a Gaussian process with time series implementation by using the slippage estimation while the rover is moving. Then, it is predicted when to change from moving to stationary conditions by mapping the predicted slippage into localization uncertainty prediction. Instead of a periodic stopping framework, the method introduced in this work is a slip-aware localization method that enables the rover to stop more frequently in high-slip terrains whereas stops rover less frequently for low-slip terrains while keeping the proprioceptive localization reliable

Autonomous Soil Assessment System for Planetary Rovers

Planetary rovers face mobility hazards associated with various classes of terrains they traverse, and hence it is desirable to enable remote prediction of terrain trafficability (ability to traverse) properties. For that reason, the development of algorithms for assessing terrain type and mobility properties, as well as for coupling these data in an online learning framework, represent important capabilities for next-generation rovers. This work focuses mainly on 3-way terrain classification (classifying as one of the types: Sand, Bedrock and Gravel) as well as on the correlation of terrain types and their mobility properties in a framework that enables online learning. For terrain classification, visual descriptors are developed, which are primarily based on visual texture and are captured in form of histograms of edge filter responses at various scales and orientations. The descriptors investigated in this work are HOG (Histogram of Oriented Gradients), GIST, MR8 (Maximum Response) Textons and the classification techniques implemented here are nearest and k-nearest neighbors. Further, monochrome image intensity is used as an additional feature to further distinguish bedrock from the other terrain types. No major differences in performance are observed between the three descriptors, leading to the adoption of the HOG approach due to its lower computational complexity (over 3 orders of magnitude difference in complexity between HOG and Textons) and thus higher applicability to planetary missions. Tests demonstrate an accuracy between 70% and 93% (81% average) for the classification using the HOG descriptor, on images taken by NASA’s Mars rovers. To predict terrain trafficability ahead of the rover, exteroceptive data namely terrain type and slope, are correlated with the trafficability metrics namely slip, sinkage and roughness, in a learning framework. A queue based data structure has been implemented for the correlation, which keeps discarding the older data so as to avoid diminishing the effect of newer data samples, when there is a large amount of data. This also ensures that the rover will be able to adapt to changing terrains responses and predict the risk level (low, medium or high) accordingly. Finally, all the algorithms developed in this work were tested and verified in a field test demo at the CSA (Canadian Space Agency) mars yard. The risk metric in combination with the queue based data structure, can achieve stable predictions in consistent terrains, while also being responsive to sudden changes in terrain trafficability

Autonomous Behaviors With A Legged Robot

Over the last ten years, technological advancements in sensory, motor, and computational capabilities have made it a real possibility for a legged robotic platform to traverse a diverse set of terrains and execute a variety of tasks on its own, with little to no outside intervention. However, there are still several technical challenges to be addressed in order to reach complete autonomy, where such a platform operates as an independent entity that communicates and cooperates with other intelligent systems, including humans. A central limitation for reaching this ultimate goal is modeling the world in which the robot is operating, the tasks it needs to execute, the sensors it is equipped with, and its level of mobility, all in a unified setting. This thesis presents a simple approach resulting in control strategies that are backed by a suite of formal correctness guarantees. We showcase the virtues of this approach via implementation of two behaviors on a legged mobile platform, autonomous natural terrain ascent and indoor multi-flight stairwell ascent, where we report on an extensive set of experiments demonstrating their empirical success. Lastly, we explore how to deal with violations to these models, specifically the robot\u27s environment, where we present two possible extensions with potential performance improvements under such conditions