Machine Learning in Robotic Navigation:Deep Visual Localization and Adaptive Control

The work conducted in this thesis contributes to the robotic navigation field by focusing on different machine learning solutions: supervised learning with (deep) neural networks, unsupervised learning, and reinforcement learning.First, we propose a semi-supervised machine learning approach that can dynamically update the robot controller's parameters using situational analysis through feature extraction and unsupervised clustering. The results show that the robot can adapt to the changes in its surroundings, resulting in a thirty percent improvement in navigation speed and stability.Then, we train multiple deep neural networks for estimating the robot's position in the environment using ground truth information provided by a classical localization and mapping approach. We prepare two image-based localization datasets in 3D simulation and compare the results of a traditional multilayer perceptron, a stacked denoising autoencoder, and a convolutional neural network (CNN). The experiment results show that our proposed inception based CNNs without pooling layers perform very well in all the environments. Finally, we propose a two-stage learning framework for visual navigation in which the experience of the agent during exploration of one goal is shared to learn to navigate to other goals. The multi-goal Q-function learns to traverse the environment by using the provided discretized map. Transfer learning is applied to the multi-goal Q-function from a maze structure to a 2D simulator and is finally deployed in a 3D simulator where the robot uses the estimated locations from the position estimator deep CNNs. The results show a significant improvement when multi-goal reinforcement learning is used

Robotic Olfactory-Based Navigation with Mobile Robots

Robotic odor source localization (OSL) is a technology that enables mobile robots or autonomous vehicles to find an odor source in unknown environments. It has been viewed as challenging due to the turbulent nature of airflows and the resulting odor plume characteristics. The key to correctly finding an odor source is designing an effective olfactory-based navigation algorithm, which guides the robot to detect emitted odor plumes as cues in finding the source. This dissertation proposes three kinds of olfactory-based navigation methods to improve search efficiency while maintaining a low computational cost, incorporating different machine learning and artificial intelligence methods. A. Adaptive Bio-inspired Navigation via Fuzzy Inference Systems. In nature, animals use olfaction to perform many life-essential activities, such as homing, foraging, mate-seeking, and evading predators. Inspired by the mate-seeking behaviors of male moths, this method presents a behavior-based navigation algorithm for using on a mobile robot to locate an odor source. Unlike traditional bio-inspired methods, which use fixed parameters to formulate robot search trajectories, a fuzzy inference system is designed to perceive the environment and adjust trajectory parameters based on the current search situation. The robot can automatically adapt the scale of search trajectories to fit environmental changes and balance the exploration and exploitation of the search. B. Olfactory-based Navigation via Model-based Reinforcement Learning Methods. This method analogizes the odor source localization as a reinforcement learning problem. During the odor plume tracing process, the belief state in a partially observable Markov decision process model is adapted to generate a source probability map that estimates possible odor source locations. A hidden Markov model is employed to produce a plume distribution map that premises plume propagation areas. Both source and plume estimates are fed to the robot. A decision-making model based on a fuzzy inference system is designed to dynamically fuse information from two maps and balance the exploitation and exploration of the search. After assigning the fused information to reward functions, a value iteration-based path planning algorithm solves the optimal action policy. C. Robotic Odor Source Localization via Deep Learning-based Methods. This method investigates the viability of implementing deep learning algorithms to solve the odor source localization problem. The primary objective is to obtain a deep learning model that guides a mobile robot to find an odor source without explicating search strategies. To achieve this goal, two kinds of deep learning models, including adaptive neuro-fuzzy inference system (ANFIS) and deep neural networks (DNNs), are employed to generate the olfactory-based navigation strategies. Multiple training data sets are acquired by applying two traditional methods in both simulation and on-vehicle tests to train deep learning models. After the supervised training, the deep learning models are verified with unseen search situations in simulation and real-world environments. All proposed algorithms are implemented in simulation and on-vehicle tests to verify their effectiveness. Compared to traditional methods, experiment results show that the proposed algorithms outperform them in terms of the success rate and average search time. Finally, the future research directions are presented at the end of the dissertation

Map-less Navigation: A Single DRL-based Controller for Robots with Varied Dimensions

Deep reinforcement learning (DRL) has shown great potential in training control agents for map-less robot navigation. However, the trained agents are generally dependent on the employed robot in training or dimension-specific, which cannot be directly reused by robots with different dimensional configurations. To address this issue, a DRL-based dimension-variable robot navigation method is proposed in this paper. The proposed approach trains a meta-agent with DRL and then transfers the meta-skill to a robot with a different dimensional configuration (named dimension-scaled robot) using a method named dimension-variable skill transfer (DVST). During the training phase, the meta-agent learns to perform self-navigation with the meta-robot in a simulation environment. In the skill-transfer phase, the observations of the dimension-scaled robot are transferred to the meta-agent in a scaled manner, and the control policy generated by the meta-agent is scaled back to the dimension-scaled robot. Simulation and real-world experimental results indicate that robots with different sizes and angular velocity bounds can accomplish navigation tasks in unknown and dynamic environments without any retraining. This work greatly extends the application range of DRL-based navigation methods from the fixed dimensional configuration to varied dimensional configurations.Comment: 10 pages, 15 figure